IBM Support

IT31677: A symptom is high latch wait times on SQLO_LT_sqeLocalDatabase__dblatch causing slow connections in automated HADR.

Subscribe

You can track all active APARs for this component.

 

APAR status

  • Closed as fixed if next.

Error description

  • As of v111m4fp5, connections may be slower in an automated HADR
    environment compared to the same environment on a previous
    fixpack. A symptom is high latch wait times on
    SQLO_LT_sqeLocalDatabase__dblatch reported in
    MON_GET_EXTENDED_LATCH_WAIT and waiters on that latch in "db2pd
    -latches" output when new connections are being opened.
    
    
    Backtraces from Db2 engine (db2sysc) collected during the issue
    with "db2pd -stack all" or OS tools ("gstack" on Linux,
    "procstack" on AIX) will show the thread holding
    SQLO_LT_sqeLocalDatabase__dblatch latch with
    sqlhaSetupHAInfrastructure function near the top of the stack,
    e.g.:
    
     read
     sqloReadNamedPipe
     sqloInitializeVendorProcess
     sqlhaSetupHAInfrastructure
     sqlhaIsHADRAutomationEnabled
     hdrCheckDb
     sqeLocalDatabase::SubsequentConnect
     sqeDBMgr::StartUsingLocalDatabase
     sqeApplication::AppStartUsing
     sqleProcessConnectType
    
     whilst thread waiting on the latch will wait in
    sqeDBMgr::StartUsingLocalDatabase->sqloSpinLockConflict, e.g.:
    
     sqloSpinLockConflict
     sqeDBMgr::StartUsingLocalDatabase
     sqeApplication::AppStartUsing
     sqleProcessConnectType
     sqeApplication::AppLocalStart
     sqlelostWrp
     sqleUCengnInit
     sqleUCagentConnect
    
    
    Example Holder Stack:
    
    *** Stack ***
    
    Timestamp: 1579190584.884346000
    0x00007F14503B2566
    _Z25ossDumpStackTraceInternalmR11OSSTrapFileiP7siginfoPvmm +
    0x0356
    0x00007F14503B21AB ossDumpStackTraceV98 + 0x002b
    0x00007F14503AE397 _ZN11OSSTrapFile6dumpExEmiP7siginfoPvm +
    0x00d7
    0x00007F1457C36497 sqlo_trce + 0x03c7
    0x00007F1457DA7F5B sqloDumpDiagInfoHandler + 0x020b
    0x00007F145F998630 address: 0x00007F145F998630 ; dladdress:
    0x00007F145F989000 ; offset in lib: 0x000000000000F630 ;
    0x00007F145F99775B read + 0x002b
    0x00007F1457C6AA71 sqloReadNamedPipe + 0x0211
    0x00007F1457C2F0FF sqloInitializeVendorProcess + 0x0e5f
    0x00007F14520309D2 address: 0x00007F14520309D2 ; dladdress:
    0x00007F145090C000 ; offset in lib: 0x00000000017249D2 ;
    0x00007F1451FEDBDD
    _Z26sqlhaSetupHAInfrastructurePP18SQLO_VENDOR_HANDLEPPciP27sqlha
    OpenDomainHandleStructPij + 0x01ad
    0x00007F145204CC61 _Z28sqlhaIsHADRAutomationEnabledPKcPb +
    0x0391
    0x00007F1451F5BE2A
    _Z10hdrCheckDbP16sqeLocalDatabaseP8sqeAgentP5sqlca + 0x0aaa
    0x00007F145607FF4B
    _ZN16sqeLocalDatabase17SubsequentConnectEP8SQLE_BWAP8sqeAgenthPi
    P8sqlo_gmti + 0x0b2b
    0x00007F1456046DDF
    _ZN8sqeDBMgr23StartUsingLocalDatabaseEP8SQLE_BWAP8sqeAgentRccP8s
    qlo_gmtPb + 0x144f
    0x00007F1455F6E2DE
    _ZN14sqeApplication13AppStartUsingEP8SQLE_BWAP8sqeAgentccP5sqlca
    Pc + 0x044e
    0x00007F1455F8144C
    _Z22sqleProcessConnectTypeP14db2UCinterfaceP8SQLE_BWAP8sqeAgentP
    5sqlcaP22SQLELOST_STATUS_VECTOR + 0x00ac
    0x00007F1455F6C970
    _ZN14sqeApplication13AppLocalStartEP14db2UCinterface + 0x07b0
    
    
    Example Waiter Stack
    Stack:
    ======================
    
    0x00007F14503B2566
    _Z25ossDumpStackTraceInternalmR11OSSTrapFileiP7siginfoPvmm +
    0x0356
      0x00007F14503B21AB ossDumpStackTraceV98 + 0x002b
      0x00007F14503AE397 _ZN11OSSTrapFile6dumpExEmiP7siginfoPvm +
    0x00d7
      0x00007F1457C36497 sqlo_trce + 0x03c7
      0x00007F1457DA7F5B sqloDumpDiagInfoHandler + 0x020b
      0x00007F145F998630 address: 0x00007F145F998630 ; dladdress:
    0x00007F145F989000 ; offset in lib: 0x000000000000F630 ;
      0x00007F145F997E9D __nanosleep + 0x002d
      0x00007F1457C3145F sqloSpinLockConflict + 0x05cf
      0x00007F145604BEC4 address: 0x00007F145604BEC4 ; dladdress:
    0x00007F145090C000 ; offset in lib: 0x000000000573FEC4 ;
      0x00007F1456046C65
    _ZN8sqeDBMgr23StartUsingLocalDatabaseEP8SQLE_BWAP8sqeAgentRccP8s
    qlo_gmtPb + 0x12d5
      0x00007F1455F6E2DE
    _ZN14sqeApplication13AppStartUsingEP8SQLE_BWAP8sqeAgentccP5sqlca
    Pc + 0x044e
      0x00007F1455F8144C
    _Z22sqleProcessConnectTypeP14db2UCinterfaceP8SQLE_BWAP8sqeAgentP
    5sqlcaP22SQLELOST_STATUS_VECTOR + 0x00ac
      0x00007F1455F6C970
    _ZN14sqeApplication13AppLocalStartEP14db2UCinterface + 0x07b0
      0x00007F145630B26F _Z11sqlelostWrpP14db2UCinterface + 0x004f
      0x00007F1456311121 _Z14sqleUCengnInitP14db2UCinterfacet +
    0x0891
      0x00007F145630777C sqleUCagentConnect + 0x089c
      0x00007F14565AE06D
    _Z18sqljsConnectAttachP13sqljsDrdaAsCbP14db2UCinterface + 0x02ed
      0x00007F145653DBFB
    _Z16sqljs_ddm_accsecP14db2UCinterfaceP13sqljDDMObject + 0x008b
      0x00007F1456531DBD
    _Z17sqljsParseConnectP13sqljsDrdaAsCbP13sqljDDMObjectP14db2UCint
    erface + 0x01ed
      0x00007F1456532FDE
    _Z10sqljsParseP13sqljsDrdaAsCbP14db2UCinterfaceP8sqeAgentb +
    0x054e
      0x00007F1456525EBD address: 0x00007F1456525EBD ; dladdress:
    0x00007F145090C000 ; offset in lib: 0x0000000005C19EBD ;
      0x00007F145652C4A3 address: 0x00007F145652C4A3 ; dladdress:
    0x00007F145090C000 ; offset in lib: 0x0000000005C204A3 ;
      0x00007F145652D15F _Z17sqljsDrdaAsDriverP18SQLCC_INITSTRUCT_T
    + 0x011f
      0x00007F1455F53CE3 _ZN8sqeAgent6RunEDUEv + 0x0de3
      0x00007F145988463E _ZN9sqzEDUObj9EDUDriverEv + 0x01be
      0x00007F1457DA5A0A sqloEDUEntry + 0x057a
      0x00007F145F990EA5 address: 0x00007F145F990EA5 ; dladdress:
    0x00007F145F989000 ; offset in lib: 0x0000000000007EA5 ;
      0x00007F144F3B68CD clone + 0x006d
    

Local fix

  • No local fix available
    

Problem summary

  • ****************************************************************
    * USERS AFFECTED:                                              *
    * Automated HADR environments                                  *
    ****************************************************************
    * PROBLEM DESCRIPTION:                                         *
    * See Error Description                                        *
    ****************************************************************
    * RECOMMENDATION:                                              *
    * Request special build if affected.                           *
    ****************************************************************
    

Problem conclusion

Temporary fix

Comments

APAR Information

  • APAR number

    IT31677

  • Reported component name

    DB2 FOR LUW

  • Reported component ID

    DB2FORLUW

  • Reported release

    B10

  • Status

    CLOSED FIN

  • PE

    NoPE

  • HIPER

    NoHIPER

  • Special Attention

    NoSpecatt / Xsystem

  • Submitted date

    2020-01-28

  • Closed date

    2020-04-03

  • Last modified date

    2020-04-03

  • APAR is sysrouted FROM one or more of the following:

  • APAR is sysrouted TO one or more of the following:

    IT36194

Fix information

Applicable component levels

[{"Line of Business":{"code":"LOB10","label":"Data and AI"},"Business Unit":{"code":"BU053","label":"Cloud & Data Platform"},"Product":{"code":"SSEPGG","label":"Db2 for Linux, UNIX and Windows"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"11.1"}]

Document Information

Modified date:
12 March 2021