Prepare the z/OS Connect native server for FFDC

When you have installed your z/OS® Connect native server, prepare it to capture vital diagnostic information if a problem occurs.

If a problem occurs, IBM® Software Support might ask you to provide diagnostic information in the form of Java™ dumps, z/OS dumps, or TDUMPS. Prepare your system now to ensure you can capture this information at the first occurrence of the problem. This is called First Failure Data Capture or FFDC.

Java dumps

To find out where different Java dumps are written, specify the following parameter in the z/OS Connect native server JVM options file, or in

STDENV
JVM_OPTIONS=

of the server's job card:

-Xdump:what

This causes the JVM to write output details of the dump configuration to the z/OS Connect native server job's STDERR:


Registered dump agents                                    
---------------------- 
-Xdump:system:
    events=gpf+user+abort+traceassert+corruptcache,
    dsn=%uid.JVM.%job.D%y%m%d.T%H%M%S.X&DS,
    range=1..0,
    priority=999,
    request=serial
----------------------                                    
-Xdump:system:                                            
    events=systhrow,
    filter=java/lang/OutOfMemoryError,
    dsn=%uid.JVM.%job.D%y%m%d.T%H%M%S.X&DS,
    range=1..1,                                           
    priority=999,
    request=exclusive+compact+prepwalk
----------------------                                    
-Xdump:heap:                                              
    events=systhrow,                                      
    filter=java/lang/OutOfMemoryError,
    file=/u/<userID>/heapdump.%Y%m%d.%H%M%S.%pid.%seq.phd,
    range=1..4,                                           
    priority=500,                                         
    request=exclusive+compact+prepwalk,
    opts=PHD
----------------------
-Xdump:java:                                              
    events=gpf+user+abort+traceassert+corruptcache,
    file=/u/<userID>/javacore.%Y%m%d.%H%M%S.%pid.%seq.txt,
    range=1..0,
    priority=400,                                         
    request=exclusive                                     
----------------------                                    
-Xdump:java:                                              
    events=systhrow,                                      
    filter=java/lang/OutOfMemoryError,
    file=/u/<userID>/javacore.%Y%m%d.%H%M%S.%pid.%seq.txt,
    range=1..4,                                           
    priority=400,                                         
    request=exclusive                                     
----------------------                                    
-Xdump:snap:                                              
    events=gpf+abort+traceassert+corruptcache,
    file=/u/<userID>/Snap.%Y%m%d.%H%M%S.%pid.%seq.trc,
    range=1..0,                                           
    priority=300,                                         
    request=serial                                        
----------------------                                    
-Xdump:snap:                                              
    events=systhrow,
    filter=java/lang/OutOfMemoryError,
    file=/u/<userID>/Snap.%Y%m%d.%H%M%S.%pid.%seq.trc, 
    range=1..4,                                           
    priority=300,
    request=serial                                        
----------------------                                    
-Xdump:jit:                                             
    events=gpf+abort,                                   
    file=/u/<userID>/jitdump.%Y%m%d.%H%M%S.%pid.%seq.dmp,
    range=1..0,                                         
    priority=200,                                       
    request=serial 
----------------------

The uid is the OMVS uid specified in the OMVS segment of the RACF® user ID. Note the uid should be unique to a single user ID.

Using this information from your system, you can confirm that the user ID that the z/OS Connect native server server is running under, has the required permissions to write all the related Java dumps.

z/OS dumps

Specify SYSMDUMP DD, SYSABEND DD, and SYSUDUMP DD cards on the z/OS Connect native server job so that system dumps are written.

TDUMPS

The default location for Java TDUMPs is %uid.JVM.TDUMP.%job.D%y%m%d.T%H%M%S

Confirm that dumps can be written

It is possible to invoke some types of dump that the z/OS Connect native server might request from SDSF. Use this method to confirm that the dumps can be written before a failure occurs. To confirm that the z/OS Connect native server can take the dump, run these commands under the user ID that starts the z/OS Connect native server job.

Ensure TDUMPs can be written

Enter the following MVS command from SDSF: /F <job_name>,TDUMP

Check the job's output for messages:

JVMDUMP034I User requested System dump using '<user_ID>.JVM. <job_name>.D200813.T153740.X&DS' through JVMRI
IEATDUMP in progress with options SDATA=(LPA,GRSQ,LSQA,NUC,PSA,RGN,SQA,SUM,SWA,TRT)
IEATDUMP success for DSN='<user_ID>.JVM.BAQ30JOB.D200813.T153740.X&DS'
JVMDUMP040I System dump written to dataset(s) using name template <user_ID>.JVM.<job_name>.D200813.T153740.X&DS

Ensure Javacore dumps can be written

Enter the following MVS command from SDSF: /F <job_name>,JAVACORE

Check the job's output for the following messages:

JVMDUMP034I User requested Java dump using <userID>/javacore.20200805.160557.67305723.0002.txt' through com.ibm.jvm.Dump.javaDumpToFile
JVMDUMP010I Java dump written to <userID>/javacore.20200805.160557.67305723.0002.txt

If the javacore dump cannot be written, the following messages might be seen:

JVMDUMP030W Cannot write dump to file <userID>/javacore.20200805.160557.67305723.0002.txt: EDC5141I Read-only file system.
JVMDUMP030W Cannot write dump to file /javacore.20150713.155831.11044.0002.txt: Permission denied

Ensure Heap dumps can be written

Enter the following MVS command from SDSF: /F <job_name>,HEAPDUMP

Check the job's output for messages:

JVMDUMP034I User requested Heap dump using '/u/<user_ID>/heapdump.202008 14.131628.16974145.0001.phd' through com.ibm.jvm.Dump.heapDumpToFile
CWWKB0005I: COMMAND RESPONSES COMPLETED SUCCESSFULLY FROM Heapdump Command Handler.
CWWKB0002I: MODIFY COMMAND HEAPDUMP COMPLETED SUCCESSFULLY.

Ensure that an MVS system dump can be written

Issue the following MVS modify commands from SDSF to request a system dump:

/DUMP COMM=(<dump_title>)
/R n,JOBNAME=(<jobname>,OMVS),CONT
/R n,SDATA=(LPA,GRSQ,LSQA,NUC,PSA,RGN,
/R n,SQA,SUM,SWA,TRT),END

Check the system log for messages:


IEA794I SVC DUMP HAS CAPTURED:
DUMPID=004 REQUESTED BY JOB (*MASTER*)
DUMP TITLE=<dump_title>
IEA611I COMPLETE DUMP ON <dump_dataset_name>
DUMPID=004 REQUESTED BY JOB (*MASTER*)
FOR ASIDS(nnnn,nnnn)

Dump related environment variables

Heap dumps

_CEE_DMPTARG specifies the location where the heap dump is written.

TDUMP

JAVA_DUMP_TDUMP_PATTERN can be used to change the naming pattern of the TDUMP.

IBM_JAVA_ZOS_TDUMP=NO prevents TDUMPS being written.

IBM_JAVA_ZOS_TDUMP_COUNT= limits the number of TDUMPS a job can request.

If IBM_JAVA_ZOS_TDUMP_PATTERN= specifies an invalid value, TDUMPs are written to STDERR.

Note: The IBM_JAVA_ZOS_TDUMP JAVA environment variable is deprecated with Java V8.

Java dumps

Check JAVA_DUMP_OPTS, Command line, and -Xdump output.

DISABLE_JAVADUMP=TRUE suppresses the default production of Java dumps.

IBM_JAVADUMP_OUTOFMEMORY=FALSE disables Java dumps for an out-of-memory exception. If not set, a Java dump is generated when an out-of-memory exception is thrown but not caught and handled by the application. Set to TRUE to generate a dump when an out-of-memory exception is thrown,

The TMPDIR=<directory> variable specifies an alternative temporary directory. This directory is used only when Java dumps and Heap dumps cannot be written to their target directories, or the current working directory. The default is /tmp.