Technical Blog Post
Abstract
-1879048166 db2diag.log error when db2fodc -hang
Body
When you issued "db2fodc -hang", it might just finish without any data collection.
In this execution, if you find out -1879048166 error in db2diag.log, it's worth checking the nofile(AIX) or open files(Linux) in "ulimit -Hn" output. These values should be larger than MAXFILOP database configuration parameter. For detail, please check the following link.
If OS is linux, you should also check if the Korn shell is installed or not.
The same error message will be return without Korn shell in linux OS.
Let me show an example for this issue.
1. -1879048166 in db2diag.log
2017-09-22-10.27.05.582112+600 I111022E261 LEVEL: Event
PID : 62187 TID : 140737352877952PROC : db2fodc
INSTANCE: xxxxxxx NODE : 000
FUNCTION: DB2 UDB, RAS/PD component, pdDb2FODCMain, probe:30
START : db2fodc started
2017-09-22-10.27.07.659853+600 I111284E343 LEVEL: Error
PID : 62187 TID : 140737352877952PROC : db2fodc
INSTANCE: xxxxxxx NODE : 000
FUNCTION: DB2 UDB, RAS/PD component, pdSwitchSplitDiagpathForDbp, probe:20
MESSAGE : ZRC=0xFFFFFFFF=-1
DATA #1 : String, 38 bytes
Subdirectory does not start with NODE.
2017-09-22-10.27.07.660764+600 E111628E667 LEVEL: Warning
PID : 62187 TID : 140737352877952PROC : db2fodc
INSTANCE: xxxxxxx NODE : 000
FUNCTION: DB2 UDB, RAS/PD component, pdDb2FODCMain, probe:30
MESSAGE : ADM14003W FODC has been invoked by the user from db2fodc tool for
symptom "hang" and diagnostic information has been recorded in
directory
"/apps/home/xxxxxxx/sqllib/db2dump/FODC_Hang_2017-09-22-10.27.07.659
766_0000". Please look in this directory for detailed evidence about
what happened and contact IBM support if necessary to diagnose the
problem.
2017-09-22-10.27.07.712841+600 I112296E310 LEVEL: Event
PID : 62187 TID : 140737352877952PROC : db2fodc
INSTANCE: xxxxxxx NODE : 000
FUNCTION: DB2 UDB, trace services, pdInvokeCalloutScriptForFODC, probe:10
START : Invoking /apps/home/xxxxxxx/sqllib/bin/db2cos_hang
2017-09-22-10.27.07.759529+600 I112607E428 LEVEL: Error (OS)
PID : 62187 TID : 140737352877952PROC : db2fodc
INSTANCE: xxxxxxx NODE : 000
FUNCTION: DB2 UDB, trace services, pdInvokeCalloutScriptForFODC, probe:100
OSERR : -1879048166 "Unknown error -1879048166"
MESSAGE : Error invoking pdInvokeCalloutScriptForFODC
DATA #1 : String, 42 bytes
/apps/home/xxxxxxx/sqllib/bin/db2cos_hang
2017-09-22-10.27.07.760021+600 I113036E262 LEVEL: Event
PID : 62187 TID : 140737352877952PROC : db2fodc
INSTANCE: db2inst2 NODE : 000
FUNCTION: DB2 UDB, RAS/PD component, pdDb2FODCMain, probe:50
STOP : db2fodc finished
2. "open files" is smaller than MAXFILOP
$ cat DBXX_node0_db.supp_cfg
...
Max DB files open per application (MAXFILOP) = 61440
...
$ cat ulimit_a.out
User Limits, 'ulimit -a'
...
open files (-n) 1024
...
[{"Business Unit":{"code":"BU058","label":"IBM Infrastructure w\/TPS"},"Product":{"code":"SSEPGG","label":"Db2 for Linux, UNIX and Windows"},"Component":"","Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"","Edition":"","Line of Business":{"code":"LOB10","label":"Data and AI"}}]
UID
ibm13286191