IBM Support

IT21587: DB2 TAKEOVER HADR COMMAND FAILS WITH SQL1387W IN AUTOMATED HADR ENVIRONMENT

Subscribe

You can track all active APARs for this component.

 

APAR status

  • Closed as program error.

Error description

  • The DB2 takeover HADR command may fail with sql code SQL1387W in
    an automated HADR environment, e.g.:
    
    SQL1387W  A DB2 High Availability Disaster Recovery (HADR)
    takeover completed successfully.  However, a HADR resource group
    remains locked; and the cluster manager is not controlling that
    resource group.
    
    The following db2diag.log errors will also be seen on the host
    where the takeover command is issued from if this issue is
    encountered:
    
    2017-06-13-05.25.57.766628+120 E365388A3020         LEVEL: Error
    PID     : 21889476             TID : 38044          PROC :
    db2sysc 0
    INSTANCE: db2inst1               NODE : 000           DB   :
    SAMPLE
    APPHDL  : 0-20353              APPID:
    *LOCAL.db2inst1.170613032415
    AUTHID  : db2inst1               HOSTNAME: test
    EDUID   : 38044                EDUNAME: db2agent (SAMPLE) 0
    FUNCTION: DB2 UDB, high avail services,
    sqlhaWaitForResourceState, probe:16314
    DATA #1 : String, 26 bytes
    db2_db2inst1_db2inst1_SAMPLE-rs
    DATA #2 : String, 0 bytes
    Object not dumped: Address: 0x0A0000009E3D2924 Size: 0 Reason:
    Zero-length data
    DATA #3 : signed integer, 4 bytes
    17
    DATA #4 : signed integer, 4 bytes
    1
    DATA #5 : SQLHA Control Block, PD_TYPE_SQLHA_CONTROL_BLOCK, 4160
    bytes
    sqlhaCB->timeout: 82
    sqlhaCB->runMode: INFRASTRUCTURE
    sqlhaCB->options: NONE
    sqlhaCB->vendorHandle: 000000011e4ed5c0
    sqlhaCB->sqlhaClusterHandle->clusterHandle: 1
    sqlhaCB->sqlhaClusterHandle->clusterFlags: 0
    sqlhaCB->sqlhaClusterHandle->clusterErrorNum: 0
    sqlhaCB->sqlhaClusterHandle->errorMessage:
    sqlhaCB->sqlhaClusterHandle->clusterCommand:
    DATA #6 : ZRC, PD_TYPE_ZRC, 4 bytes
    0x87000057
    DATA #7 : SQLHA Cluster Session Handle,
    PD_TYPE_SQLHA_CLUSTER_HANDLE, 4120 bytes
    sqlhaClusterHandle->clusterHandle: 1
    sqlhaClusterHandle->clusterFlags: 0
    sqlhaClusterHandle->clusterErrorNum: 0
    sqlhaClusterHandle->errorMessage:
    sqlhaClusterHandle->clusterCommand:
    DATA #8 : signed integer, 4 bytes
    0
    CALLSTCK: (Static functions may not be resolved correctly, as
    they are resolved to the nearest symbol)
      [0] 0x0900000011605B2C
    sqlhaWaitForResourceState__FP25SQLHA_CLUSTER_OBJECT_INFO15_sqlha
    ObjStatesP19SQLHA_CONTROL_BLOCK + 0x540
      [1] 0x090000000C55B894
    sqlhaEnableClusterAutomationForHADRResource__FP16sqeLocalDatabas
    eP19SQLHA_CONTROL_BLOCK + 0xED4
      [2] 0x090000000E2FA930
    sqlerKnownProcedure__FiPcPiP5sqldaT4P13sqlerFmpTableP8sqeAgentP5
    sqlca + 0x43290
      [3] 0x090000000E2F0724
    sqlerKnownProcedure__FiPcPiP5sqldaT4P13sqlerFmpTableP8sqeAgentP5
    sqlca + 0x39084
      [4] 0x090000000E2207FC
    sqlerKnownProcedure__FiPcPiP5sqldaT4P13sqlerFmpTableP8sqeAgentP5
    sqlca + 0xF54
      [5] 0x090000000E201914
    sqlerCallDL__FP14db2UCinterfaceP9UCstpInfo + 0x604
      [6] 0x090000000FF30864
    sqljsParseRdbAccessed__FP13sqljsDrdaAsCbP13sqljDDMObjectP14db2UC
    interface + 0x644
      [7] 0x090000000FF198E0
    @72@sqljsSqlam__FP14db2UCinterfaceP8sqeAgentb + 0x11E4
      [8] 0x090000000FF198E0
    @72@sqljsSqlam__FP14db2UCinterfaceP8sqeAgentb + 0x11E4
      [9] 0x090000000FF194D4
    @72@sqljsSqlam__FP14db2UCinterfaceP8sqeAgentb + 0xDD8
      [10] 0x0900000010C874E4
    @72@sqljsDriveRequests__FP8sqeAgentP14db2UCconHandle + 0xA8
      [11] 0x0900000010C87FF8
    @72@sqljsDrdaAsInnerDriver__FP18SQLCC_INITSTRUCT_Tb + 0x5FC
      [12] 0x09000000106D795C RunEDU__8sqeAgentFv + 0x40E1C
      [13] 0x090000001064C738 RunEDU__8sqeAgentFv + 0x124
      [14] 0x090000000F461AE4 EDUDriver__9sqzEDUObjFv + 0x130
      [15] 0x090000000F0F32D4 sqloEDUEntry + 0x3A0
      [16] 0x0900000000509E10 _pthread_body + 0xF0
      [17] 0xFFFFFFFFFFFFFFFC ?unknown + 0xFFFFFFFF
    
    2017-06-13-05.25.57.768189+120 E368409A719          LEVEL: Error
    PID     : 21889476             TID : 38044          PROC :
    db2sysc 0
    INSTANCE: db2inst1               NODE : 000           DB   :
    SAMPLE
    APPHDL  : 0-20353              APPID:
    *LOCAL.db2inst1.170613032415
    AUTHID  : db2inst1               HOSTNAME: test
    EDUID   : 38044                EDUNAME: db2agent (SAMPLE) 0
    FUNCTION: DB2 UDB, high avail services, sqlhaEnableHADRResource,
    probe:14174
    MESSAGE : ZRC=0x87000057=-2030043049=SQLZ_RC_TIMEOUT "Action
    timed out"
              DIA8578C A timeout occurred while waiting on a
    semaphore.
    DATA #1 : String, 47 bytes
    Unable to verify HADR resource state as online.
    DATA #2 : String, 26 bytes
    db2_db2inst1_db2inst1_SAMPLE-rs
    
    2017-06-13-05.25.57.799078+120 E369129A524          LEVEL: Error
    PID     : 21889476             TID : 38044          PROC :
    db2sysc 0
    INSTANCE: db2inst1               NODE : 000           DB   :
    SAMPLE
    APPHDL  : 0-20353              APPID:
    *LOCAL.db2inst1.170613032415
    AUTHID  : db2inst1               HOSTNAME: test
    EDUID   : 38044                EDUNAME: db2agent (SAMPLE) 0
    FUNCTION: DB2 UDB, high avail services, sqlhaUnLockHADRResource,
    probe:14927
    RETCODE : ZRC=0x827300AA=-2106392406=HA_ZRC_FAILED "SQLHA API
    call error"
    
    2017-06-13-05.25.57.799700+120 E369654A800          LEVEL: Error
    PID     : 21889476             TID : 38044          PROC :
    db2sysc 0
    INSTANCE: db2inst1               NODE : 000           DB   :
    SAMPLE
    APPHDL  : 0-20353              APPID:
    *LOCAL.db2inst1.170613032415
    AUTHID  : db2inst1               HOSTNAME: test
    EDUID   : 38044                EDUNAME: db2agent (SAMPLE) 0
    FUNCTION: DB2 UDB, high avail services, sqlhaUnLockHADRResource,
    probe:14927
    MESSAGE : ZRC=0x827300AA=-2106392406=HA_ZRC_FAILED "SQLHA API
    call error"
    DATA #1 : String, 6 bytes
    db2inst1
    DATA #2 : String, 6 bytes
    db2inst1
    DATA #3 : String, 5 bytes
    SAMPLE
    DATA #4 : String, 44 bytes
    db2_SAMPLE_ClusterInitiatedMove_db2inst1_db2inst1
    DATA #5 : String, 0 bytes
    Object not dumped: Address: 0x0A0000009E3F42F4 Size: 0 Reason:
    Zero-length data
    
    2017-06-13-05.25.57.800569+120 I370455A574          LEVEL:
    Warning
    PID     : 21889476             TID : 38044          PROC :
    db2sysc 0
    INSTANCE: db2inst1               NODE : 000           DB   :
    SAMPLE
    APPHDL  : 0-20353              APPID:
    *LOCAL.db2inst1.170613032415
    AUTHID  : db2inst1               HOSTNAME: test
    EDUID   : 38044                EDUNAME: db2agent (SAMPLE) 0
    FUNCTION: DB2 UDB, High Availability Disaster Recovery,
    hdrTakeoverHdrRouteIn, probe:55620
    RETCODE : ZRC=0x827300D7=-2106392361=HA_ZRC_HDR_UNLOCK_FAILED
              "Unlock HADR Resource Group failed"
    

Local fix

  • n/a
    

Problem summary

  • ****************************************************************
    * USERS AFFECTED:                                              *
    * Automated HADR users                                         *
    ****************************************************************
    * PROBLEM DESCRIPTION:                                         *
    * See Error Description                                        *
    ****************************************************************
    * RECOMMENDATION:                                              *
    * Upgrade to Db2 V11.1.3.3                                     *
    ****************************************************************
    

Problem conclusion

  • Resolved in Db2 V11.1.3.3
    

Temporary fix

Comments

APAR Information

  • APAR number

    IT21587

  • Reported component name

    DB2 FOR LUW

  • Reported component ID

    DB2FORLUW

  • Reported release

    B10

  • Status

    CLOSED PER

  • PE

    NoPE

  • HIPER

    NoHIPER

  • Special Attention

    NoSpecatt / Xsystem

  • Submitted date

    2017-07-25

  • Closed date

    2018-03-15

  • Last modified date

    2018-03-15

  • APAR is sysrouted FROM one or more of the following:

    IT21327

  • APAR is sysrouted TO one or more of the following:

Fix information

  • Fixed component name

    DB2 FOR LUW

  • Fixed component ID

    DB2FORLUW

Applicable component levels

  • RB10 PSN

       UP

[{"Business Unit":{"code":"BU053","label":"Cloud & Data Platform"},"Product":{"code":"SSEPGG","label":"Db2 for Linux, UNIX and Windows"},"Component":"","ARM Category":[],"Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"11.1","Edition":"","Line of Business":{"code":"LOB10","label":"Data and AI"}}]

Document Information

Modified date:
15 March 2018