IBM Support

MustGather: Using the -Xdump Option

Troubleshooting


Problem

Dump agents are set up during JVM initialization. They enable you to use events occurring within the JVM, such as Garbage Collection, thread start, or JVM termination, to initiate one of four types of dump or to launch an external tool. Default dump agents are set up at JVM initialization They are sufficient for most cases, but the use of the -Xdump option on the command line allows more detailed configuration of dump agents. The total set of options and sub-options available under -Xdump is very flexible and there are many examples presented in this chapter to show this flexibility.

Resolving The Problem

The -Xdump option allows you add and remove dump agents for various JVM events, update default dump settings (such as the dump name), and limit the number of dumps that are produced.

Below is described:







Help options:
You can obtain help on the various usage aspects of -Xdump by using:


java -Xdump:help



which produces:
Usage:
 -Xdump:help            Print general dump help
 -Xdump:none            Ignore all previous and default dump options
 -Xdump:events          List available trigger events
 -Xdump:request         List additional VM requests
 -Xdump:tokens          List recognized label tokens
 -Xdump:dynamic         Enable support for pluggable agents
 -Xdump:what            Show registered agents on startup

 -Xdump:<type>:help     Print detailed dump help
 -Xdump:<type>:none     Ignore previous dump options of this type
 -Xdump:<type>:defaults Print and update default settings for this type
 -Xdump:<type>          Request this type of dump (using defaults)

Dump types:

 -Xdump:console         Basic thread dump to stderr
 -Xdump:system          Capture raw process image.
 -Xdump:tool            Run command line program
 -Xdump:java            Write application summary.
 -Xdump:heap            Capture heap graph.
 -Xdump:snap            Take a snap of the trace buffers

Example:

java -Xdump:heap:none -Xdump:heap:events=fullgc class [args...]

turns off default Heapdumps and then requests a Heapdump on every full GC.

As can be seen from the usage section, further help is available for the assorted sub-options under -Xdump. In particular:

java -Xdump:events

The preceding command shows the available keywords used to specify the events that can be used. You must filter class events (such as load, throw, and uncaught) by class name.

Supported event keywords 	 Event hook
 gpf ON_GP_FAULT
 user   ON_USER_SIGNAL
 abort ON_ABORT_SIGNAL
 vmstart ON_VM_STARTUP
 vmstop ON_VM_SHUTDOWN
 load ON_CLASS_LOAD
 unload ON_CLASS_UNLOAD
 throw ON_EXCEPTION_THROW
 catch ON_EXCEPTION_CATCH
 brkpoint ON_BREAKPOINT
 framepop ON_DEBUG_FRAME_POP
 thrstart ON_THREAD_START
 blocked ON_THREAD_BLOCKED
 thrstop ON_THREAD_END
 expand ON_HEAP_EXPAND
 fullgc ON_GLOBAL_GC
 uncaught ON_EXCEPTION_DESCRIBE
 slow ON_SLOW_EXCLUSIVE_ENTER
 any *


Dump types and triggering:
The main purpose of the -Xdump stanza on the command line is to link events to a dump type (-Xdump:tool is a little misleading, because it is a command, not a dump). Thus, -Xdump:heap:events=vmstop is an instruction to JVM initialization to create a dump agent that produces a Heapdump whenever the vmstop event happens. The JVM is constructed to generate at the appropriate time the events listed in Using core (system) dumps.

You can have multiple -Xdump stanzas on the command line and also multiple dump types driven by one or multiple events. Thus, -Xdump:heap+java:events=vmstart+vmstop would create a dump agent that would drive both heap and Java dump production when either a vmstart or vmstop event was encountered.

Note that multiple -Xdump stanzas on the command line can be used to create multiple agents at JVM initialization; these agents are chained together and all evaluated whenever an event occurs. The dump agent processing ensures that multiple -Xdump stanzas are optimized. You can use the -Xdump:what stanza to clarify this optimization.

The keyword events is used as the prime trigger mechanism. However, there are a number of additional keywords that you can use to further control the dump produced (request and tokens, for example) or limit its production to a smaller range of circumstances; use -Xdump:help to find these.


Types of dump agents - examples:
This section presents several examples of the use of -Xdump, based around each dump type, to illustrate the style of syntax and the generated function. The examples given are deliberately simplistic to limit the size of the output.

As you can see from using -Xdump:help, there are many dump types to consider.
  1. Console Dumps
    Console dumps are simple dumps, in which the status of every Java thread is written to stderr. Some output of this type is shown below. Note the use of the range=1..1 suboption to control the amount of output to just one thread start and stop (in this case, the start of the Signal Dispatcher thread).

    java -Xdump:console:events=thrstart+thrstop,range=1..1
    JVMDUMP006I Processing Dump Event "thrstart", detail "" - Please Wait.
    -------- Console dump --------
    Stack Traces of Threads:
    ThreadName=Signal Dispatcher(00035B24)
    Status=Running
    ThreadName=main(00035A1C)
    Status=Waiting
    Monitor=00035128 (VM sig quit)
    Count=0
    Owner=(00000000)
    ^^^^^^^^ Console dump ^^^^^^^^

  2. System dumps
    System dumps involve dumping a whole frozen address space and as such are generally very large. The bigger the footprint of an application the bigger its dump. A dump of a major server-based application might take up many gigabytes of file space and take several minutes to complete. Shown below is an example of invoking a system dump on a Windows 32-bit machine. Note the use of request=nodumps+exclusive+prepwalk in this example, to ensure that this dump is not interrupted by other dumps and that the Java heap is walkable, enabling the objects within the heap to be processed under jextract or jdmpview (the equivalent of -Xdump:heapdump in the previous release). Note also that the file name is overridden from the default in this example.

    java -Xdump:system:events=vmstop,request=nodumps+exclusive+prepwalk,file=my.dmp
    ::::::::: removed usage info :::::::::  

    JVMDUMP006I Processing Dump Event "vmstop", detail "#00000000" - Please Wait.
    JVMDUMP007I JVM Requesting System Dump using 'C:\sdk\sdk\jre\bin\my.dmp'
    JVMDUMP010I System Dump written to C:\sdk\sdk\jre\bin\my.dmp
    JVMDUMP013I Processed Dump Event "vmstop", detail "#00000000".

  3. Tool option
    : The tool option allows external processes to be spawned when an event occurs. Consider the following simple example, which displays start of pid and end of pid information (note the use of the token %pid). More realistic examples would invoke a debugging tool, and that is the default taken if you use (for example)

    -Xdump:tool:events=.....
    java -Xdump:tool:events=vmstop,exec="cmd /c echo %pid has finished"
    -Xdump:tool:events=vmstart,exec="cmd /c echo %pid has started"

    JVMDUMP006I Processing Dump Event "vmstart", detail "" - Please Wait.
    JVMDUMP007I JVM Requesting Tool Dump using 'cmd /c echo 2184 has started'
    JVMDUMP011I Tool Dump spawned process 2160
    2184 has started
    JVMDUMP013I Processed Dump Event "vmstart", detail "".

    ::::::::: removed usage info :::::::::

    JVMDUMP006I Processing Dump Event "vmstop", detail "#00000000" - Please Wait.
    JVMDUMP007I JVM Requesting Tool Dump using 'cmd /c echo 2184 has finished'
    JVMDUMP011I Tool Dump spawned process 2204
    2184 has finished
    JVMDUMP013I Processed Dump Event "vmstop", detail "#00000000".

  4. Javadumps:
    Java dumps are an internally generated and formatted analysis of the JVM, giving information that includes the Java threads present, the classes loaded, and heap statistics. An example (which also shows the use of the filter keyword) in which a Javadump is produced on the loading of a class is shown below.

    java -Xdump:java:events=load,filter=*String
    JVMDUMP006I Processing Dump Event "load", detail "java/lang/String" - Please Wait.
    JVMDUMP007I JVM Requesting Java Dump using
     C:\sdk\jre\bin\javacore.20051012.162700.2836.txt'
    JVMDUMP010I Java Dump written to
     C:\sdk\jre\bin\javacore.20051012.162700.2836.txt
    JVMDUMP013I Processed Dump Event "load", detail "java/lang/String".

  5. Heapdumps:
    From Version 1.4.2, Service Refresh 2, Heapdumps now produce phd format files by default unless overridden. The example below shows the production of a Heapdump. Note that in this case the normal production of a phd file only has been augmented by the use of the opts= suboption to produce both phd and classic (.txt) heapdumps (equivalent to using the environment variable IBM_JAVA_HEAPDUMP_TEST).

    java -Xdump:none -Xdump:heap:events=vmstop,opts=PHD+CLASSIC
    JVMDUMP006I Processing Dump Event "vmstop", detail "#00000000" - Please Wait.
    JVMDUMP007I JVM Requesting Heap Dump using
     'C:\sdk\jre\bin\heapdump.20050323.142011.3272.phd'
    JVMDUMP010I Heap Dump written to
     C:\sdk\jre\bin\heapdump.20050323.142011.3272.phd
    JVMDUMP007I JVM Requesting Heap Dump using
     'C:\sdk\jre\bin\heapdump.20050323.142011.3272.txt'
    JVMDUMP010I Heap Dump written to
     C:\sdk\jre\bin\heapdump.20050323.142011.3272.txt
    JVMDUMP013I Processed Dump Event "vmstop", detail "#00000000".

  6. Snap traces:
    Version 5.0 added snap traces to the range of outputs controlled by -Xdump. The example below shows the production of a snap trace.

    java -Xdump:none -Xdump:snap:events=vmstop+vmstart
    JVMDUMP006I Processing Dump Event "vmstart", detail "" - Please Wait.
    JVMDUMP007I JVM Requesting Snap Dump using
     'C:\sdk\jre\bin\Snap0001.20051012.161706.2804.trc'
    JVMDUMP010I Snap Dump written to
     C:\sdk\jre\bin\Snap0001.20051012.161706.2804.trc
    JVMDUMP013I Processed Dump Event "vmstart", detail "".

    Usage: java [-options] class [args...]
              (to execute a class)
      ====  extraneous lines removed for terseness ====
        -assert   print help on assert options

    JVMDUMP006I Processing Dump Event "vmstop", detail "#00000000" - Please Wait.
    JVMDUMP007I JVM Requesting Snap Dump using
     'C:\sdk\jre\bin\Snap0002.20051012.161706.2804.trc'
    JVMDUMP010I Snap Dump written to
     C:\sdk\jre\bin\Snap0002.20051012.161706.2804.trc
    JVMDUMP013I Processed Dump Event "vmstop", detail "#00000000".
Notice how the snap traces are given sequential numbers (Snap0001 then |Snap0002). The produced snap traces are encoded and require the use of the |trace formatter for further analysis.


Default dump agents:
The JVM adds a set of dump agents by default during its initialization. You can override this set of dump agents using the JAVA_DUMP_OPTS environment variable and further override the set by the use of -Xdump on the command line.

The -Xdump:what option on the command line is very useful for determining which dump agents exist at the completion of startup and can help resolve issues about what has overridden what. Below is sample output showing the default dump agents that are in place when there have been no overrides by using environment variables.

java -Xdump:what
Registered dump agents
----------------------
dumpFn=doSystemDump
events=gpf+abort
filter=
label=C:\sdk\jre\bin\core.%Y%m%d.%H%M%S.%pid.%seq.dmp
range=1..0
priority=999
request=serial
opts=
----------------------
dumpFn=doSnapDump
events=gpf+abort
filter=
label=C:\sdk\jre\bin\Snap%seq.%Y%m%d.%H%M%S.%pid.%seq.trc
range=1..0
priority=500
request=serial
opts=
----------------------
dumpFn=doSnapDump
events=systhrow
filter=java/lang/OutOfMemoryError
label=C:\sdk\jre\bin\Snap%seq.%Y%m%d.%H%M%S.%pid.%seq.trc
range=1..4
priority=500
request=serial
opts=
----------------------
dumpFn=doHeapDump
events=systhrow
filter=java/lang/OutOfMemoryError
label=C:\sdk\jre\bin\heapdump.%Y%m%d.%H%M%S.%pid.%seq.phd
range=1..4
priority=40
request=exclusive+compact+prepwalk
opts=PHD
----------------------
dumpFn=doJavaDump
events=gpf+user+abort
filter=
label=C:\sdk\jre\bin\javacore.%Y%m%d.%H%M%S.%pid.%seq.txt
range=1..0
priority=10
request=exclusive
opts=
----------------------
dumpFn=doJavaDump
events=systhrow
filter=java/lang/OutOfMemoryError
label=C:\sdk\jre\bin\javacore.%Y%m%d.%H%M%S.%pid.%seq.txt
range=1..4
priority=10
request=exclusive
opts=
----------------------

Default settings for dumps:
To view the default settings for a particular dump type, use:

    -Xdump:<type>:defaults

You can change these defaults at runtime. For example, you can direct Java dump files into a separate directory for each process, and guarantee unique files by adding a sequence number to the file name using:

    -Xdump:java:defaults:file=dumps/%pid/javacore-%seq.txt

Or, for example, on z/OS, you can add the jobname to the Java dump file name using:

    -Xdump:java:defaults:file=javacore.%job.%H%M%S.txt

This option does not add a javadump agent; it updates the default settings for dump agents. Further dump agents will then create dump files using this specification for filenames, unless overridden.


Limiting dumps using filters and range keywords:
Some JVM events occur thousands of times during the lifetime of an application. Dump agents can use filters and ranges to avoid excessive dumps being produced.

You can filter class events (such as load, throw, and uncaught) by class name:

    -Xdump:java:events=throw,filter=java/lang/OutOfMem* # prefix
   -Xdump:java:events=throw,filter=*MemoryError # suffix
   -Xdump:java:events=throw,filter=*Memory* # substring

You can filter the JVM shutdown event by using one or more exit codes:

    -Xdump:java:events=vmstop,filter=#129..192#-42#255

You can start and stop dump agents on a particular occurrence of a JVM event by using the range suboption:

    -Xdump:java:events=fullgc,range=100..200

Note that range=1..0 against an event means "on every occurrence".


Removing dump agents:
You can remove all default dump agents and any preceding dump options by using:

    -Xdump:none

Use this option so that you can subsequently specify a completely new dump configuration.
You can also remove dump agents of a particular type. For example,

    -Xdump:java+heap:events=vmstop -Xdump:heap:none

turns off all heapdumps (including default agents) but leaves javadump enabled.


Controlling dump ordering:
In some situations, one event can generate multiple dumps. For example, examination of the output from java -Xdump:what shows that a gpf event produces a snap trace, a java dump, and a system dump. The agents that produce each dump run sequentially and their order is determined by the priority keyword set for each agent. In the example, the system dump would run first (priority 999), the snap dump second (priority 500), and the javadump last (priority 10). An example of setting the priority from the command line is:

java -Xdump:heap:events=vmstop,priority=123

The maximum value allowed for priority is 999 and the higher the priority the higher a dump agent will sit in the chain.

If you do not specifically set a priority then default values are taken based on the dump type. The default priority, as well as the other default values for a particular type of dump, can be displayed by using the defaults option on the -Xdump invocation, for example:

C:\sdk\jre\bin>java -Xdump:heap:defaults

Default -Xdump:heap settings:

events=gpf+user
filter=
file=C:\sdk\jre\bin\heapdump.%Y%m%d.%H%M%S.%pid.phd
range=1..0
priority=40
request=exclusive+prepwalk
opts=PHD


Controlling dump file names:
Dumps are created by default in the working directory. Most often, this directory is the one from which the application was launched. If the dump cannot be created there for any reason (such as it being a read-only location), alternative locations are tried.

The -Xdump:what option on the command line is very useful for determining the default dump location. Below is part of the sample output showing the default dump location that is in place when there have been no overrides by using environment variables.

java -Xdump:what
Registered dump agents
----------------------
dumpFn=doSystemDump
events=gpf+abort
filter=
label=C:\sdk\jre\bin\core.%Y%m%d.%H%M%S.%pid.dmp
range=1..0
priority=999
request=serial
opts=

----------------------

The pattern in the file setting for that particular dump agent determines the name of the dump. The defaults (use java -Xdump:<type>:defaults) are usually sufficient; however, you can use the file keyword on the command line to set your own file name and location as following:

java -Xdump:heap:events=vmstop,file=my.dmp

In z/OS platforms, to generate the system dumps, use the dsn option instead of the file option.For example:

java -Xdump:system:events=vmstop,dsn=%uid.MYDUMP

[{"Product":{"code":"SSNVBF","label":"Runtimes for Java Technology"},"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Component":"Debugging Options","Platform":[{"code":"PF002","label":"AIX"},{"code":"PF016","label":"Linux"},{"code":"PF033","label":"Windows"},{"code":"PF035","label":"z\/OS"}],"Version":"5.0","Edition":"J2SE","Line of Business":{"code":"LOB36","label":"IBM Automation"}}]

Document Information

Modified date:
15 June 2018

UID

swg21242497