Troubleshooting
Problem
MustGather for problems with WebSphere Application Server hang problems during the running of an Application Server which was previously successfully processing requests. Gathering the information before calling IBM support will help familiarize you with the troubleshooting process and save you time.
Resolving The Problem
This MustGather covers the scenario when an Application Server is successfully processing requests and all of a sudden hangs. There is a separate MustGather for the scenario when you have issued the stop command or tried a stop via the Administrative Console but the server you are requesting to stop is not coming down or hangs. That MustGather is titled "MustGather: A hang occurs when attempting to stop WebSphere Application Server for z/OS".
MustGather information for the specific problem when a hang occurs in an Application Server which was previously processing work:
If you have not contacted support click: MustGather: Read first for WebSphere Application Server for z/OS link.
- Obtain the trace
Ensure that the Trace output is being written to sysprint using this MVS modify command:
f control_region_name,tracetosysprint=yes
Note: When setting the trace, case is important in the tracejava keyword. If using SDSF to enter commands, use the SDSF extended command line (by putting a slash, / , on the command line and press enter); otherwise SDSF makes the string uppercase.
Issue MVS modify commands:
f control_region_name,tracejava='com.ibm.*=all=enabled'
f control_region_name,tracedetail=(3,4)
Attempt to send some more requests to this server.
Let the trace run for 30 seconds.
Issue the following MVS modify command to turn off the trace:
f control_region_name,traceinit
- Dump hanging address spaces
At a minimum, take a console dump of the WebSphere Application Server Controller and Servant(s) involved in the hang, along with OMVS, RRS and WLM address spaces and the OMVS data spaces.
DUMP COMM=(Descriptive name for this WebSphere dump)
R rn,SDATA=(ALLNUC,CSA,GRSQ,LPA,LSQA,PSA,RGN,SQA,SUM,SWA,TRT),CONT
R rn,JOBNAME=(OMVS,WLM,RRS,control_region_name, servant_region_name),CONT
R rn,DSPNAME=('OMVS'.*),END
If you feel other resource managers, such as DB2 or IMS or MQ, were involved in this hang, please take a console dump of these address spaces as well.
The WebSphere Application Server, OMVS, RRS, and WLM address spaces should all be able to be contained in one console dump, as long as there are not too many Servant Regions. However, if other resource managers are involved, those should be taken as a separate console dump.
Ensure that any console dumps are not partial dumps. Verify that you see message IEA611I in the SYSLOG. This message indicates that a Complete dump was taken.
- Send documentation
FTP the Control and Servant Region joblogs.
FTP the tersed console dump(s) in binary format.
Review the information in the following link before sending documentation to IBM:
Submitting Diagnostic Information to IBM.
For a listing of all technotes search the WebSphere Application Server for z/OS support site.
Related Information
Was this topic helpful?
Document Information
Modified date:
15 June 2018
UID
swg21181842