What data should I collect for a HANG situation with DB2 For Linux, Unix and Windows?

Answer

Disclaimer - DB2 Universal Database (all versions) is a very complex product with many different components interacting with one another. This being the case, it is not possible to outline all the diagnostic data collections that may be needed to resolve any/all problems that may arise. Each issue with DB2 is unique, and in most cases, there is an initial round of data that will provide a good overall view of all aspects of DB2's behavior in a given circumstance. While this may be enough to identify root cause, it is not uncommon for this initial round of data to indicate a problem in a specific area, and that a second round of data is required to dig deeper into that.

In addition to the diagnostic data outlined below, DB2 support will need to have a detailed description of the problem. They will ask many questions to get an overall picture of the problem seen. During this time, additional diagnostic data may be requested. Also, after reviewing the initial set of data, additional diagnostics may be required.

Due to this fact, it would be best if a PMR could be opened before collecting data, so the support analyst can get some details about the issue, and then perhaps tailor the initial data collection so as to minimize the chances of additional collections being required. However there still is no guarantee that additional data collections will not be required.

Must-gather for a HANG data collection:

A DB2 hang is defined as a situation where DB2, or a portion of DB2, is no longer responding to inputs, or is no longer producing output, or otherwise processing. The principal tool that is used to gather data to analyze a hang is the db2fodc (DB2 First Occurrence Data Capture) command.

If a hang is observed, then you should (as the instance owner) log in to the DB2 server itself, and execute the following command:

db2fodc -hang full

This command will take some time to complete, but it will gather information crucial to the root cause analysis. If there is more than one PHYSICAL node for this instance, please execute the command on each PHYSICAL node.

It will produce a directory called "FODC_Hang_XXXXXXX" in the db2dump (DIAGPATH) of the DB2 server/node, where XXXXXXX is the timestamp that the command was executed.

Once the command completes, the FODC information can be gathered using the db2support command as follows:

db2support . -s

Again, if there is more than one PHYSICAL node, then the command should be executed on all PHYSICAL nodes.

If the system is hung, then it may not be possible to execute this command (it may hang as well) until the system is restarted. However if possible, you should open a PMR (Severity 1, down system) and discuss the issue with support before restarting to ensure that no additional data is needed before restarting.

If this is not possible (IE need to get the system back up ASAP), then the system can now be restarted (Please ensure that the db2fodc hang command has completed first) using whatever method works best (IE db2_kill, ipclean, etc). Then once it is back up, collect the db2support information.

Sending in the data:

The collected data should be uploaded to the associated PMR. If there is no PMR open for the issue yet, then please open one first. Then once you have the PMR number, there are a couple of ways to upload the data.

Before uploading, please ensure that the data is named appropriately. If DB2 support has asked for data to be named a certain way, then please do so. If they have not, then please come up with a good naming scheme. For example, if you have multiple db2support.zip files from multiple machines (IE DPF environment with multiple physical nodes), then you can name them like:

db2support.<hostname>.zip

1. You can use a browser to upload the data. Go to

http://www.ecurep.ibm.com/app/upload

Then follow the directions there.

2. Command line FTP. You can go here:

http://www.ibm.com/support/pages/node/739283

For directions to upload data. Please be sure to follow the directions under "Prepare your data" to ensure that the upload and attachment to the PMR is successful, or there may be delays in support gaining access to it.