IBM on troubleshooting Java applications

Debugging the cause of Java exceptions is a common activity for Java developers, however this is not always easy to do as the exception messages are often generic. A good example of this is the java.net.ConnectException. When trying to make a socket connection to a host that is not accepting connections, you get the following message:

    java.net.ConnectException: Connection refused: connect
    	at java.net.PlainSocketImpl.socketConnect(Native Method)
    	at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:352)
    	at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:214)
    	at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:201)
    	at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:377)
    	at java.net.Socket.connect(Socket.java:530)
    	at java.net.Socket.connect(Socket.java:480)
    	at java.net.Socket.(Socket.java:377)
    	at java.net.Socket.(Socket.java:220)

This is sufficient if you have access to the code that's creating the socket, and you can see from the code which hostname and port is being used. In the case of more complex code where the values used for the hostname and port are subject to change as they are obtained from external sources (user input values, databases etc), this doesn't help you to understand why the connection is being refused.

Getting more information when the exception is generated
The Java Runtimes from IBM provide (in Java 5.0 and 6.0) a mechanism for triggering additional data generation on various events using the -Xdump option. On these events its possble to generate several different dump types which can provide some insight as to the cause of the exception.

What additional data can be generated?
The -Xdump options allow you to generate a number of different dump types when the trigger event occurs, however the following are most likely to be of use for debugging exceptions:

Dump type	Data generated
stack	produces a stack trace for the thread that caused the event.
java	produces a javacore text file, which provides a full thread dump along with environment, class loading and heap usage overview information
system	produces a full memory image of the Java process in the form of a non-destructive system dump (core file, svcdump etc)

The "stack" option may seen unnecessary as you get a stack trace for an exception anyway, but it can be useful if there is logging code in use that is removing or truncating the stack trace. Of the other two dump types, "java" can be useful if you want to look at what the other threads are doing when the exception occurred, and a core file is useful if you want to see the values stored in objects associated with the exception.

Which trigger events are available?
Available events include the "throw" and "catch" of exceptions, as well as when exceptions are left "uncaught". This means that its possible to set up a dump trigger to generate more data when the java.net.ConnectException is produced.

It is usually best to use the the "throw" event rather than "catch" or "uncaught" as the catch and uncaught events occur after the stack has been unwound/rolled back, which means that the stack trace represents the point at which the catch block in the code is hit rather than the point the exception is generated. As this is after the initial generation of the exception, objects may have been released and threads may have moved on.

Putting it all together
In order to set up the generation of a system dump on a ConnectException, you can use the following added to the command line:

    -Xdump:system:events=throw,range=1..1,filter=java/net/ConnectException

This creates a "system" dump on the event of exception "thrown" with a filter to generate dumps for only exceptions of type "java/net/ConnectException". Note that the filter uses a slash form of the exception name rather than the more usual dot form.

Also note that there is a "range" set to "1..1". This means that only a single dump will be generated. As exceptions can be and often are re-thrown several times, this is required to prevent multiple dumps being generated.

To achieve the same thing but produce a javacore.txt file, you just need to change the "system" value to "java":

    -Xdump:java:events=throw,range=1..1,filter=java/net/ConnectException

Analysing the new data
Now that you can generate extra data for the exception, the next step is use that data to find our more information about the cause of the exception and to hopefully solve the problem it's reporting.

In the next entry I'll go through how to use a system dump from a java.net.ConnectException to find our what hostname and port was being used.