IBM Support

IT29736: REPLAYING DROP TABLESPACE WOULD SET POOL ID TO -1 IN FILTERED TABLESPACE LIST ON STANDBY, CAUSING SQLPRSETPOOLENDLFS() TO PANIC

Subscribe to this APAR

By subscribing, you receive periodic emails alerting you to the status of the APAR, along with a link to the fix after it becomes available. You can track this item individually or track all items by product.

Notify me when this APAR changes.

Notify me when an APAR for this component changes.

 

APAR status

  • Closed as program error.

Error description

  • Takeover from Standby may fail due to SQLB_INV_POOLID. This can
    happen when any tablespace is in abnormal state on Standby.
    
    Diaglog Messages -
    
    The tablespace 8 was in abnormal state on standby and is added
    to filter list:
    
    2019-07-05-17.08.58.370467-300 I56314965A570        LEVEL:
    Warning
    PID     : 49808374             TID : 69664          PROC :
    db2sysc 0
    INSTANCE: db2inst2             NODE : 000           DB   :
    SAMPLE
    APPHDL  : 0-652                APPID: *LOCAL.DB2.190705220858
    HOSTNAME: Standby
    EDUID   : 69664                EDUNAME: db2agent (SAMPLE) 0
    FUNCTION: DB2 UDB, recovery manager,
    sqlpGetTablespacesForFilter, probe:1570
    DATA #1 : <preformatted>
    Table space MONITOR (8) is in rollforward pending state.
    Another rollforward will be needed to bring this tablespace
    online.
    
    2019-07-05-17.08.58.371027-300 I56315536A544        LEVEL: Info
    PID     : 49808374             TID : 69664          PROC :
    db2sysc 0
    INSTANCE: db2inst2             NODE : 000           DB   :
    SAMPLE
    APPHDL  : 0-652                APPID: *LOCAL.DB2.190705220858
    HOSTNAME: Standby
    EDUID   : 69664                EDUNAME: db2agent (SAMPLE) 0
    FUNCTION: DB2 UDB, data protection services,
    sqlpMasterDbcb::sqlpAddTbspToAbnormalList, probe:20
    MESSAGE : Added tablespace to abnormal tablespace list.
    DATA #1 : Pool ID, PD_TYPE_SQLB_POOL_ID, 2 bytes
    8
    
        2019-07-08-13.19.30.816522-300 I46756A434          LEVEL:
    Info
        PID    : 49808374            TID : 69400         PROC :
    db2sysc 0
        INSTANCE: db2inst2            NODE : 000          DB  :
    SAMPLE
        HOSTNAME: Standby
        EDUID  : 69400               EDUNAME: db2hadrs.0.0 (SAMPLE)
    0
        FUNCTION: DB2 UDB, High Availability Disaster Recovery,
    hdrStbyTkHandleInitialRequest, probe:46000
        MESSAGE : Standby has initiated a takeover by force.
    
        2019-07-08-13.19.31.861422-300 I61567A7659         LEVEL:
    Severe
        PID    : 49808374            TID : 69664         PROC :
    db2sysc 0
        INSTANCE: db2inst2            NODE : 000          DB  :
    SAMPLE
        APPHDL : 0-652               APPID: *LOCAL.DB2.190705220858
        HOSTNAME: Standby
        EDUID  : 69664               EDUNAME: db2agent (SAMPLE) 0
        FUNCTION: <0>, <0>, <0>, probe:1382
        MESSAGE : ZRC=0x87020076=-2029911946=SQLB_INV_POOLID
    "Invalid pool ID"
        DATA #1 : String, 112 bytes
        sqlbGetExistingPoolDef: Error getting pool definition after
    retrying. The caller may need to use sqlbGetPoolDef.
        DATA #2 : Pool ID, PD_TYPE_SQLB_POOL_ID, 2 bytes
        65535
        DATA #3 : SQLB_PTBL, PD_TYPE_SQLB_PTBL, 1160 bytes
         poolCount:                            35
         maxPoolID:                            34
         ptbl_flags:                            1
         pool_table_latch:
        {
          state        = 0x0000000000000000
                        = {
                              held X: 0
                              reserved for X: 0
                              shared holders: 0
                              firstSharIndex: 0x0
                              firstExclIndex: 0x0
                          }
          starve X mode = true
          xWaitCount   = 0
          requestCount = 0
          identity     = SQLB_PTBL::pool_table_latch (142)
        }
         DisablePoolCount:                      1
         numSysTempPools:
        0x0A00030001233724 : 0000 0000 0000 0000 0000 0000 0000 0002
    ................
         defaultSysTempPool:
        0x0A00030001233734 : FFFF FFFF FFFF FFFF FFFF FFFF 0000 0001
    ................
         dropPendingList:      0x0000000000000000
         suspendingList:       0x0000000000000000
         suspendAction:                         0
         containersToDelete:   0x0a0003000120eba0
         poolCBs:
        0x0A00030001233768 : 0A00 0300 0121 0820 0000 0000 0000 0000
    .....!. ........
    
         maxIdInGrowableArrays:               511
        CALLSTCK: (Static functions may not be resolved correctly,
    as they are resolved to the nearest symbol)
         [0] 0x0900000008872598
    sqlbPutEndLfs__FP12SQLB_GLOBALSP18SQLB_POOL_STATE_IN + 0x3458
         [1] 0x09000000092A50DC
    sqlprSetPoolEndLfs__FP8sqeAgentCUsCUl + 0xDC
         [2] 0x090000000C5E42FC
    sqlpForwardRecovery__FP8sqeAgentUsPCcPiPUiT2 + 0x377C
         [3] 0x090000000CA995A8 sqlpReplayMaster__FP8sqeAgent +
    0x1DC8
         [4] 0x090000000A431C8C
    sqleIndCoordProcessRequest__FP8sqeAgent + 0x10CC
         [5] 0x090000000794A8E8 RunEDU__8sqeAgentFv + 0xAA8
         [6] 0x0900000007924BC0 EDUDriver__9sqzEDUObjFv + 0x300
         [7] 0x0900000007806200 sqloEDUEntry + 0x360
         [8] 0x090000000055DE10 _pthread_body + 0xF0
         [9] 0xFFFFFFFFFFFFFFFC ?unknown + 0xFFFFFFFF
    
        2019-07-08-13.19.31.863054-300 E69227A945          LEVEL:
    Critical
        PID    : 49808374            TID : 69664         PROC :
    db2sysc 0
        INSTANCE: db2inst2            NODE : 000          DB  :
    SAMPLE
        APPHDL : 0-652               APPID: *LOCAL.DB2.190705220858
        HOSTNAME: Standby
        EDUID  : 69664               EDUNAME: db2agent (SAMPLE) 0
        FUNCTION: DB2 UDB, RAS/PD component, pdStartFODC, probe:10
        MESSAGE : ADM14001C An unexpected and critical error has
    occurred: "Panic".
                 The instance may have been shutdown as a result.
    "Automatic" FODC
                 (First Occurrence Data Capture) has been invoked
    and diagnostic
                 information has been recorded in directory
    
    "/db2inst2/SAMPLE/backup/DB2DUMP/FODC_Panic_2019-07-08-13.19.31.
    862
                 298_0000/". Please look in this directory for
    detailed evidence about
                 what happened and contact IBM support if necessary
    to diagnose the
                 problem.
    
    
    </Registers>
    <POFDisassembly>
         0x0900000000575EF4 : EB81FFE0 ld   r28,-32(r1)
         0x0900000000575EF8 : EBA1FFE8 ld   r29,-24(r1)
         0x0900000000575EFC : EBC1FFF0 ld   r30,-16(r1)
         0x0900000000575F00 : EBE1FFF8 ld   r31,-8(r1)
         0x0900000000575F04 : 4E800020 br       # 20,bit0
         0x0900000000575F08 : 3860FFFF li   r3,0xFFFF
         0x0900000000575F0C : 63840000 ori  r4,r28,0x0
         0x0900000000575F10 : 48000921 bl   0x900000000576830
    >>>> 0x0900000000575F14 : E8410028 ld   r2,40(r1)
         0x0900000000575F18 : 38600000 li   r3,0x0
         0x0900000000575F1C : EAA10098 ld   r21,152(r1)
         0x0900000000575F20 : E9810100 ld   r12,256(r1)
         0x0900000000575F24 : 382100F0 addi r1,r1,240
         0x0900000000575F28 : 7D8803A6 mtlr r12
    </POFDisassembly>
    <StackTrace>
    -------Frame------ ------Function + Offset------
    0x0900000000575F14 pthread_kill + 0xD4
    0x0900000000575764 _p_raise + 0x44
    0x0900000000039768 raise + 0x48
    0x0900000000056164 abort + 0xC4
    0x090000000780C9F8 sqloExitEDU + 0x298
    0x0900000007878F44 sqle_panic__Fi + 0x7A4
    0x09000000086DA228 sqlb_panic + 0xA8
    0x09000000088725A0
    sqlbPutEndLfs__FP12SQLB_GLOBALSP18SQLB_POOL_STATE_IN + 0x3460
    0x09000000092A50DC sqlprSetPoolEndLfs__FP8sqeAgentCUsCUl + 0xDC
    0x090000000C5E42FC sqlpForwardRecovery__FP8sqeAgentUsPCcPiPUiT2
    + 0x377C
    0x090000000CA995A8 sqlpReplayMaster__FP8sqeAgent + 0x1DC8
    0x090000000A431C8C sqleIndCoordProcessRequest__FP8sqeAgent +
    0x10CC
    0x090000000794A8E8 RunEDU__8sqeAgentFv + 0xAA8
    0x0900000007924BC0 EDUDriver__9sqzEDUObjFv + 0x300
    0x0900000007806200 sqloEDUEntry + 0x360
    </StackTrace>
    <ProcessObjects>
    

Local fix

  • The workaround is to deactivate and activate on standby after
    drop tablespace is replayed on standby.
    This way the dropped tablespace will not appear in filtered
    tablespace list and takeover will not have any problem.
    
    If the above is not then then only way is to reinitialize HADR
    on standby.
    

Problem summary

  • ****************************************************************
    * USERS AFFECTED:                                              *
    * ALL                                                          *
    ****************************************************************
    * PROBLEM DESCRIPTION:                                         *
    * See Error Description                                        *
    ****************************************************************
    * RECOMMENDATION:                                              *
    * Upgrade to Db2 11.1 Mod 4 Fixpack 5 or higher                *
    ****************************************************************
    

Problem conclusion

  • First fixed in Db2 11.1 Mod 4 Fixpack 5
    

Temporary fix

Comments

APAR Information

  • APAR number

    IT29736

  • Reported component name

    DB2 FOR LUW

  • Reported component ID

    DB2FORLUW

  • Reported release

    B10

  • Status

    CLOSED PER

  • PE

    NoPE

  • HIPER

    NoHIPER

  • Special Attention

    NoSpecatt / Xsystem

  • Submitted date

    2019-07-15

  • Closed date

    2020-01-16

  • Last modified date

    2020-01-16

  • APAR is sysrouted FROM one or more of the following:

  • APAR is sysrouted TO one or more of the following:

Fix information

  • Fixed component name

    DB2 FOR LUW

  • Fixed component ID

    DB2FORLUW

Applicable component levels

  • RB10 PSN

       UP

[{"Business Unit":{"code":"BU053","label":"Cloud & Data Platform"},"Product":{"code":"SSEPGG","label":"Db2 for Linux, UNIX and Windows"},"Component":"","ARM Category":[],"Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"11.1","Edition":"","Line of Business":{"code":"LOB10","label":"Data and AI"}}]

Document Information

Modified date:
16 January 2020