db2cos (callout script) output files
A db2cos script is invoked by default when the database manager cannot continue processing due to a panic, trap, segmentation violation or exception. Each default db2cos script will invoke db2pd commands to collect information in an unlatched manner.
The names for the db2cos scripts are in the form db2cos_hang, db2cos_trap, and so on. Each script behaves in a similar way except db2cos_hang which is called from the db2fodc tool.
The default db2cos scripts are found under the bin directory. On UNIX operating systems, this directory is read-only. You can copy the db2cos script file and all the relevant scripts to the adm directory if customization on all files is required. If a db2cos script is found in the adm directory, it is run; otherwise, the default script in the bin directory is run. For example, for data corruption issues, Db2® looks for a db2cos_datacorruption script. If there is a customized script in the adm directory, Db2 uses this customized script. Otherwise, the default db2cos_datacorruption script in the bin directory is used. For information on what scripts are called automatically and for which situation see, Collecting diagnosis information based on common outage problems.
In a multiple partition configuration, the script will only be invoked for the trapping agent on the partition encountering the trap. If it is required to collect information from other partitions, you can update the db2cos script to use the db2_all command or, if all of the partitions are on the same machine, specify the -alldbpartitionnums option on the db2pd command.
The types of signals that trigger the invocation of db2cos are also configurable via the db2pdcfg -cos command. The default configuration is for the db2cos script to run when either a panic or trap occurs. However, generated signals will not launch the db2cos script by default.
- Trap file is created
- Signal handler is called
- db2cos script is called (depending on the db2cos settings enabled)
- An entry is logged in the administration notification log
- An entry is logged in the db2diag log file
The default information collected by the db2pd command in the db2cos script includes details about the operating system, the Version and Service Level of the installed Db2 product, the database manager and database configuration, as well as information about the state of the agents, memory pools, memory sets, memory blocks, applications, utilities, transactions, buffer pools, locks, transaction logs, table spaces and containers. In addition, it provides information about the state of the dynamic, static, and catalog caches, table and index statistics, the recovery status, as well as the reoptimized SQL statements and an active statement list. If you have to collect further information, update the db2cos script with the additional commands.
When the default db2cos script is called, it produces output files in the directory specified by the DIAGPATH database manager configuration parameter. The files are named XXX.YYY.ZZZ.cos.txt, where XXX is the process ID (PID), YYY is the thread ID (TID) and ZZZ is the database partition number (or 000 for single partition databases). If multiple threads trap, there will be a separate invocation of the db2cos script for each thread. In the event that a PID and TID combination occurs more than once, the data will be appended to the file. There will be a timestamp present so you can distinguish the iterations of output.
2005-10-14-10.56.21.523659 PID : 782348 TID : 1 PROC : db2cos INSTANCE: db2inst1 NODE : 0 DB : SAMPLE APPHDL : APPID: *LOCAL.db2inst1.051014155507 FUNCTION: oper system services, sqloEDUCodeTrapHandler, probe:999 EVENT : Invoking /home/db2inst1/sqllib/bin/db2cos from oper system services sqloEDUCodeTrapHandler Trap Caught Instance db2inst1 uses 64 bits and Db2 code release SQL09010 ... Operating System Information: OSName: AIX NodeName: n1 Version: 5 Release: 2 Machine: 000966594C00 ...
2005-10-14-10.42.17.149512-300 I19441A349 LEVEL: Event PID : 782348 TID : 1 PROC : db2sysc INSTANCE: db2inst1 NODE : 000 FUNCTION: Db2, trace services, pdInvokeCalloutScript, probe:10 START : Invoking /home/db2inst1/sqllib/bin/db2cos from oper system services sqloEDUCodeTrapHandler 2005-10-14-10.42.23.173872-300 I19791A310 LEVEL: Event PID : 782348 TID : 1 PROC : db2sysc INSTANCE: db2inst1 NODE : 000 FUNCTION: Db2, trace services, pdInvokeCalloutScript, probe:20 STOP : Completed invoking /home/db2inst1/sqllib/bin/db2cos 2005-10-14-10.42.23.519227-300 E20102A509 LEVEL: Severe PID : 782348 TID : 1 PROC : db2sysc INSTANCE: db2inst1 NODE : 000 FUNCTION: Db2, oper system services, sqloEDUCodeTrapHandler, probe:10 MESSAGE : ADM0503C An unexpected internal processing error has occurred. ALL DB2 PROCESSES ASSOCIATED WITH THIS INSTANCE HAVE BEEN SHUTDOWN. Diagnostic information has been recorded. Contact IBM Support for further assistance. 2005-10-14-10.42.23.520111-300 E20612A642 LEVEL: Severe PID : 782348 TID : 1 PROC : db2sysc INSTANCE: db2inst1 NODE : 000 FUNCTION: Db2, oper system services, sqloEDUCodeTrapHandler, probe:20 DATA #1 : Signal Number Recieved, 4 bytes 11 DATA #2 : Siginfo, 64 bytes 0x0FFFFFFFFFFFD5C0 : 0000 000B 0000 0000 0000 0009 0000 0000 ................ 0x0FFFFFFFFFFFD5D0 : 0000 0000 0000 0000 0000 0000 0000 0000 ................ 0x0FFFFFFFFFFFD5E0 : 0000 0000 0000 0000 0000 0000 0000 0000 ................ 0x0FFFFFFFFFFFD5F0 : 0000 0000 0000 0000 0000 0000 0000 0000 ................