Collecting diagnosis information

Diagnostic information can be gathered automatically in a first occurrence data capture (FODC) package as the problem that affects an instance, host, or member is occurring. You can use the db2fodc command to collect performance data for issues that do not trigger automatic FODC.

Automatic collection of diagnostic information

For specific errors, the database manager automatically collects diagnostic information in the FODC package, which in turn may invoke one of the Db2® call-out scripts (COSs).

To correlate the outage with the Db2 diagnostic logs and the other troubleshooting files, a diagnostic message is written to both the administration notification and the db2diag log files. The FODC package directory name comprises the FODC_ prefix, the name of the outage type, the timestamp when the FODC directory was created, and the number of the member or partition where the problem occurred. The FODC package description file is placed in the FODC package directory.

Table 1. Automatic FODC types and packages
Package name and description	Executed script
FODC_Memory_`timestamp`_`memberName` An out-of-memory (OOM) error occurred.	db2cos.bat
FODC_SingleQuery_955_`timestamp`_`memberNumber` An operation that requires shared sort memory exceeded the database sort heap size (sortheap) and failed with SQL955N.	db2cos.bat
FODC_ ElasticSort_`timestamp`_`memberNumber` An operation that requires shared sort memory collected diagnostic information that exceeded the database sort heap size (sortheap). However, the operation completed successfully, and no action is necessary.	db2cos.bat
FODC_ SharedSort_955_`timestamp`_`memberNumber` Concurrent operations requiring shared sort memory exceeded the database sort heap threshold for shared sorts (sheapthres_shr). This caused a query to fail with SQL955N.	db2cos.bat
FODC_ WLM_Queue_`timestamp`_`memberNumber` The workload manager admission control feature queued a statement for longer than expected. This triggered an internal diagnostic process that created this FODC package. No action is required.	db2cos.bat
FODC_AppErr_`timestamp`_`PID_EDUID`_`memberNumber` An error occurred in the application. For example, there was a commit or rollback failure, an SQLCODE -901 or -902 error occurred, or an unsustainable trap occurred while running a trusted routine.	db2cos.bat activates when a sub-agent encounters a commit or rollback failure, or an SQLCODE -901 or -902 error. db2cos_trap.bat activates when an unsustainable trap occurs while running a trusted routine.
FODC_Trap_`timestamp`_`memberNumber` An instance wide trap occurred.	db2cos_trap.bat
FODC_Panic_`timestamp`_`memberNumber` Engine detected an incoherence and decided not to continue.	db2cos.bat
FODC_BadPage_`timestamp`_`memberNumber` A bad page was detected.	db2cos_datacorruption.bat
FODC_DBMarkedBad_`timestamp`_`memberNumber` A database has been marked bad due to an error.	db2cos.bat
FODC_`[Index\|Data\|Col]Error_directory`_`timestamp`_`PID_EDUID`_`memberNumber` An EDU wide index error occurred.	`db2cos_[index\|data\|col]error_long.bat` or `db2cos_[index\|data\|col]error_short.bat`
FODC_Member_`timestamp`_`memberNumber` A member or partition has failed or has received a kill signal.	db2cos_member.bat
FODC_`db2stop_timestamp` Db2stop processing entered the force phase when all applications have been forced off, but at least 2 minutes passed since the force was issued and it has not yet completed. This triggered an internal diagnostic process that created this FODC package. No action is required.	db2cos.bat
FODC_ FcmResource_`timestamp_memberNumber` During request processing, the Fast Communications Manager (FCM) ran out of buffer or channel resources. This triggered an internal diagnostic process that created this FODC package. No action is required.	db2cos.bat

Manual collection of diagnostic information

Use the db2fodc command manually when you suspect a problem is occurring. Problem scenarios that you can collect diagnostic data for include apparent system hangs, performance issues, or when an upgrade operation or instance creation did not complete as expected. When the db2fodc command is run manually, a new FODC package directory is created. The FODC package directory name includes the FODC_ prefix, the problem scenario, the timestamp when the FODC directory was created, and the number of each member or partition where FODC was performed.

Table 2. Manual FODC types and packages
Package name and description	Executed script
FODC_Clp_`timestamp`_`memberNumber` User invoked db2fodc -clp to collect environment and configuration related information, used to troubleshoot problems related to instance creation.	db2cos_clp.bat
FODC_Connections_`timestamp`_`memberNumber` User invoked db2fodc -connections to collect connection-related diagnostic data, used to diagnose problems such as sudden spikes in the number of applications in the executing or compiling state or new database connections being denied.	db2cos_threshold.bat
FODC_Cpu_`timestamp`_`memberNumber` User invoked db2fodc -cpu to collect processor-related performance and diagnostic data, used to diagnose problems such as high processor utilization rates, a high number of running processes, or high processor wait times.	db2cos_threshold.bat
FODC_Hang_`timestamp`_`memberList` User invoked db2fodc -hang to collect data for hang troubleshooting (or severe performance).	db2cos_hang.bat
FODC_Memory_`timestamp`_`memberNumber` User invoked db2fodc -memory to collect memory-related diagnostic data, used to diagnose problems such as no free memory available, swap space being used at a high rate, excessive paging or a suspected a memory leak.	db2cos_threshold.bat
FODC_Perf_`timestamp`_`memberList` User invoked db2fodc -perf to collect data for performance troubleshooting.	db2cos_perf.bat
FODC_Preupgrade_`timestamp`_`memberNumber` User invoked db2fodc -preupgrade to collect performance related information before a critical upgrade or update such as upgrading an instance or updating to the next fix pack.	db2cos_preupgrade.bat
FODC_ `[Index\|Data\|Col]Error_directory`_`timestamp`_`PID_EDUID`_`memberNumber` User could issue db2fodc -fodcerror `FODC_[Index\|Data\|Col]Error_directory` [basic \| full] (default is basic) to invoke the db2dart commands in the scripts. On partitioned database environments, use db2_all "<<+`node#`< db2fodc -fodcerror `FODC_[Index\|Data\|Col]Error_directory` [basic \| full]". The `node#` is the last number in the `FODC_[Index\|Data\|Col]Error_directory` directory name. An absolute path is required when using db2fodc -fodcerror with the db2_all command.	`db2cos_[index\|data\|col]error_long.bat` or `db2cos_[index\|data\|col]error_short.bat`