IBM Support

IT22759: DB2 AGENT/DB2FMP CAN HANG DUE TO INCOMPLETE HANDSHAKE DURING INTERRUPT

Subscribe

You can track all active APARs for this component.

 

APAR status

  • Closed as program error.

Error description

  • db2 agent/db2fmp can hang due to incomplete handshake during
    interrupt.  This can result in any agent looking to call a
    routine to get stuck waiting on the
    SQLO_LT_sqlerFmpTable__fmpTableLatch latch result in a hang for
    that agent as well.
    
    db2diag.log:
    
    2017-08-23-00.16.02.999306+000 I20742564A998        LEVEL: Error
    PID     : 2557828              TID : 23697          PROC :
    db2sysc 24
    INSTANCE: db2inst1             NODE : 024           DB   :
    DBNAME
    APPHDL  : 999-5773             APPID:
    *N999.db2inst1.170823001545
    AUTHID  : db2inst1             HOSTNAME: db2host1
    EDUID   : 23697                EDUNAME: db2agntp (DBNAME) 24
    FUNCTION: DB2 UDB, routine_infrastructure,
    sqlerInvokeFencedRoutine, probe:1220
    MESSAGE : ZRC=0x804B0003=-2142568445=SQLJS_INTRPT "INTERRUPT
    DETECTED"
              DIA8003C The interrupt  has been received.
    DATA #1 : SQLCA, PD_DB2_TYPE_SQLCA, 136 bytes
    sqlcaid : SQLCA     sqlcabc: 136   sqlcode: 0   sqlerrml: 0
    sqlerrmc:
    sqlerrp : SQL11012
    sqlerrd : (1) 0x00000000      (2) 0x00000000      (3) 0x00000000
               (4) 0x00000000      (5) 0x00000000      (6)
    0x00000018
    sqlwarn : (1)      (2)      (3)      (4)        (5)       (6)
               (7)      (8)      (9)      (10)        (11)
    sqlstate:
    
    2017-08-23-00.16.02.689867+000 I20743563A899        LEVEL: Error
    PID     : 8913656              TID : 772            PROC :
    db2fmp (C) 9
    INSTANCE: db2inst1             NODE : 009           DB   :
    DBNAME
    APPID   : *N999.db2inst1.170823000555
    HOSTNAME: db2host
    EDUID   : 772                  EDUNAME: db2fmp (C) 9
    FUNCTION: DB2 UDB, routine_infrastructure, sqlerFmpReceive,
    probe:1259
    MESSAGE : ZRC=0x8024006D=-2145124243=SQLER_CA_BUILT
              "SQLCA has already been built"
    DATA #1 : SQLCA, PD_DB2_TYPE_SQLCA, 136 bytes
    sqlcaid : SQLCA     sqlcabc: 136   sqlcode: -1131   sqlerrml: 0
    sqlerrmc:
    sqlerrp : SQLERRCV
    sqlerrd : (1) 0x00000000      (2) 0x00000000      (3) 0x00000000
               (4) 0x00000000      (5) 0x00000000      (6)
    0x00000009
    sqlwarn : (1)      (2)      (3)      (4)        (5)       (6)
               (7)      (8)      (9)      (10)        (11)
    sqlstate:
    
    2017-08-23-00.16.02.691142+000 I20744463A556        LEVEL:
    Severe
    PID     : 8913656              TID : 1              PROC :
    db2fmp (C) 9
    INSTANCE: db2inst1             NODE : 009
    HOSTNAME: db2host
    EDUID   : 1                    EDUNAME: db2fmp (C) 9
    FUNCTION: DB2 UDB, trace services, sqlt_logerr_data (secondary
    logging function), probe:50
    MESSAGE : Library still in use in sqlerLibraryUnloadAll
              /db2home/db2inst1/sqllib/bin/routine/db2monudf 3
    DATA #1 : Hexdump, 4 bytes
    0x0FFFFFFFFFFFDDB0 : 0000 0000
    ....
    
    2017-08-23-00.16.02.519850+000 E20745020A4253       LEVEL:
    Severe
    PID     : 10159080             TID : 10285          PROC :
    db2sysc 57
    INSTANCE: db2inst1             NODE : 057           DB   :
    DBNAME
    APPHDL  : 999-5773             APPID:
    *N999.db2inst1.170823001545
    AUTHID  : db2inst1             HOSTNAME: db2host
    EDUID   : 10285                EDUNAME: db2agntp (DBNAME) 57
    FUNCTION: DB2 UDB, routine_infrastructure, sqlerMasterThreadReq,
    probe:910
    DATA #1 : String, 23 bytes
    Marking fmp as unstable
    DATA #2 : String, 8 bytes
    Fmp TID:
    DATA #3 : Hexdump, 4 bytes
    0x0780000002AFFEB8 : 0000 0607
    ....
    DATA #4 : String, 8 bytes
    Fmp Row:
    DATA #5 : sqlerFmpRow, PD_SQLER_TYPE_FMP_ROW, 856 bytes
    fmpPid: 5636556
    fmpPoolList Ptr: 0x0780000002a3f740    fmpForcedList Ptr:
    0x0000000000000000
    nextFmpCB Ptr: 0x0780000000da5360      prevFmpCB Ptr:
    0x0000000000000000
    fmpIPCList Ptr: 0x0780000002a3ffc0
    stateFlags: 0x00000012 numFmp32Attaches: 0
    numActiveThreads: 1    numPoolThreads: 3
    fmpCodePage: 0 fmpRowUseCount: 1
    active: 0x01   rowLoaderValidate: 0x00
    ipcLatch:
    0x0780000002A3EB64 : 1000 0000 003F 0000
    .....?..
    rowLatch:
    0x0780000002A3EB6C : 1000 0000 01DB 0000
    ........
    fmpAgentList:
    
    2017-08-23-00.16.02.999763+000 I20755577A495        LEVEL:
    Severe
    PID     : 10421710             TID : 1              PROC :
    db2fmp (C) 24
    INSTANCE: db2inst1              NODE : 024
    HOSTNAME: db2host1
    EDUID   : 1                    EDUNAME: db2fmp (C) 24
    FUNCTION: DB2 UDB, routine_infrastructure,
    sqlerMasterThreadListener, probe:400
    RETCODE : ZRC=0xFFFFFB95=-1131
              SQL1131N  A stored procedure process has been
    terminated abnormally.
              Routine name: "". Specific name: "".
    
    Agent Stack could look like:
    
    *** Stack ***
    
    Timestamp: 2017-08-23-17.55.48.247784
    0x09000000001F5C10 semop + 0xB0
    0x090000000E118664 sqloSSemP + 0x1224
    0x0900000012C7AD14
    IPRA.$sqlccipcWaitSynch__FP18SQLCC_INITSTRUCT_T + 0x254
    0x0900000012C74254 sqlccipcinit + 0x8D4
    0x090000000E10F94C
    sqlccinit__FP18SQLCC_INITSTRUCT_TPP17SQLCC_COMHANDLE_TP12SQLCC_C
    OND_TP13SQLO_MEM_POOL + 0x5CC
    0x090000000EBF7EBC
    IPRA.$sqlerInitCommsLayer__FP14sqlerFmpHandleP8sqeAgentb + 0x11C
    0x090000000EBF34D8
    sqlerMasterThreadReq__FP17sqlerFmpParmsBaseP13sqlerFmpTableP14sq
    lerFmpHandleP18sqlerFmpThreadListP8sqeAgentUicT7P5sqlcab+ 0x858
    0x090000000EBF6340
    sqlerInterruptThreadedFmp__FP14sqlerFmpHandleP17sqlerFmpParmsBas
    e + 0xC0
    0x090000000EBE2E1C
    sqlerReturnFmpToPool__FcT1P14sqlerFmpHandleP8sqeAgent + 0x1DDC
    0x090000000EBDED24 sqlerInvokeFencedRoutine__FP13sqlerFmpParms +
    0xB44
    0x0900000011D54F54 sqlriInvokeInvoker__FP10sqlri_ufobb + 0x1CF4
    0x0900000011D56C9C sqlriutf__FP8sqlrr_cb + 0x13C
    0x09000000145156B8 sqlri_tfopn__FP8sqlrr_cbP9sqlri_tao + 0x1F8
    0x09000000101296A4 sqlriopn__FP8sqlrr_cbP9sqlri_taoPi + 0x5A4
    0x090000001204C750 sqlrita__FP8sqlrr_cb + 0xEF0
    0x090000000FCDC898 sqlriSectInvoke__FP8sqlrr_cbP12sqlri_opparm +
    0x618
    0x090000000F634380 sqlrr_dss_router__FP8sqlrr_cb + 0x9A0
    0x090000000F633054
    sqlrr_subagent_router__FP8sqeAgentP12SQLE_DB2RA_T + 0xF54
    

Local fix

Problem summary

  • ****************************************************************
    * USERS AFFECTED:                                              *
    * N/A                                                          *
    ****************************************************************
    * PROBLEM DESCRIPTION:                                         *
    * See Error Description                                        *
    ****************************************************************
    * RECOMMENDATION:                                              *
    * Upgrade to db2_v111m3fp3.                                    *
    ****************************************************************
    

Problem conclusion

  • db2_v111m3fp3 contains fix.
    

Temporary fix

Comments

APAR Information

  • APAR number

    IT22759

  • Reported component name

    DB2 FOR LUW

  • Reported component ID

    DB2FORLUW

  • Reported release

    B10

  • Status

    CLOSED PER

  • PE

    NoPE

  • HIPER

    NoHIPER

  • Special Attention

    NoSpecatt / Xsystem

  • Submitted date

    2017-10-12

  • Closed date

    2018-03-15

  • Last modified date

    2018-03-15

  • APAR is sysrouted FROM one or more of the following:

  • APAR is sysrouted TO one or more of the following:

Fix information

  • Fixed component name

    DB2 FOR LUW

  • Fixed component ID

    DB2FORLUW

Applicable component levels

  • RB10 PSY

       UP

[{"Business Unit":{"code":"BU053","label":"Cloud & Data Platform"},"Product":{"code":"SSEPGG","label":"Db2 for Linux, UNIX and Windows"},"Component":"","ARM Category":[],"Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"11.1","Edition":"","Line of Business":{"code":"LOB10","label":"Data and AI"}}]

Document Information

Modified date:
15 March 2018