IBM Support

IV95270: DB2 IS DOWN WHEN AIO FAILED TO COLLECT ALL OUTSTANDING IO REQUESTS.

Subscribe

You can track all active APARs for this component.

 

APAR status

  • Closed as fixed if next.

Error description

  • db2diag.log showed that there was still one page outstanding
    and hadn't been collected, which led  assertion failure / panic.
    2017-04-07-12.59.44.093685+480 I13323030E34369   LEVEL:Severe
    PID     : 42704                TID : 140384869017344 PROC :
    db2sysc 1
    INSTANCE: istrpt               NODE : 001            DB   :
    F12RPTB
    HOSTNAME: f12p1ddbsm02
    EDUID   : 129                  EDUNAME: db2pclnr (F12RPTB) 1
    FUNCTION: DB2 UDB, buffer pool services, sqlbClnrCollectAllAIO,
    probe:1960
    DATA #1 : String, 146 bytes
    ASSERTION FAILED!!!
    ASSERTION EXPRESSION: ((clnrCB->lioCB.numAIOOutstanding == 0))
    SOURCE FILE NAME: sqlbclnr_core.C
    SOURCE FILE LINE NUMBER: 1960
    DATA #2 : Codepath, 8 bytes
    0
    DATA #3 : SQLB_CLNR_CB, PD_TYPE_SQLB_CLNR_CB, 6480 bytes
    clnrIndex: 0
    parentGroup: 0x00007fc473c89000
    eduCB: 00007fc473cd64e0
    Cleaner Latch (clnrLatch):
    {
       lock          = { 0x00 [ unlocked ] }
       fighterCount  = 0
       identity      = SQLB_PAGE_CLEANER::clnrLatch (277)
    }
    clnrState: 2
    To Cleaner WaitPost (toClnrWP):
    {
       postcode  = 0x0
       cs        = {
       lock          = { 0x00 [ unlocked ] }
       fighterCount  = 0
       identity      = sqlo_waitpost::cs (6)
    }
       state     = 0x0 (Not Posted)
       guard     = initialized
       initflags = 0x1
       first = 0x0000000000000000
    }
    From Cleaner WaitPost (fromClnrWP):
    {
       postcode  = 0x0
       cs        = {
       lock          = { 0x00 [ unlocked ] }
       fighterCount  = 0
       identity      = sqlo_waitpost::cs (6)
    }
       state     = 0x0 (Not Posted)
       guard     = initialized
       initflags = 0x1
       first = 0x0000000000000000
    }
    SQLB_CLNR_LIO_CB:
    {
       numAIOOutstanding: 1
       maxAIO: 256
       numToCollect: 128
       lioFlags: 4
       batchIORequestList
       {
          IOReqList: 00007fc473ca91e0
          numPagesInList: 0
          lowPos: 231
          highPos: 232
          appendToHighPos: true
       }
       serializeBatchIORequestList
       {
          IOReqList: 00007fc473c88040
          numPagesInList: 0
          lowPos: 0
          highPos: 0
          appendToHighPos: false
       }
       serializeIORequestList
       {
          IOReqList: 00007fc473c5a3a0
          numPagesInList: 0
          lowPos: 0
          highPos: 0
          appendToHighPos: false
       }
       completedIOReqScratch: 00007fc473c3b3a0
    }
    -- Start of warmCB --
    0x00007FC473CAA180 : 0000 0000 0000 0000 0000 0000 0000 0000
    ................
    0x00007FC473CAA190 : 0000 0000 0000 0000 0000 0000 0000 0000
    ................
    0x00007FC473CAA1A0 : 0000 0000 0000 0000 0000 0000 0000 0000
    ................
    <skip>
    -- End of castoutCB --
    SQLB_GLOBALS: 7FC98CED5080
    terminating: false
    forcedTerminate false:
    currentGatherTask: 9
    -- Start of currentGatherInfo --
    0x00007FC473CAB750 : 0000 0000 0000 0000 0000 0000 0000 0000
    ................
    0x00007FC473CAB760 : 0000 0000 0000 0000 0000 0000 0000 0000
    ................
    0x00007FC473CAB770 : 0000 0000 0000 0000
    ........
    -- End of currentGatherInfo --
    -- Start of gatherStatics --
    0x00007FC473CAB778 : 0100 0000 0000 0000 0000 0000 0000 0000
    ................
    0x00007FC473CAB788 : 0000 0000 0000 0000 0000 0000 0000 0000
    ................
    0x00007FC473CAB798 : 0000 0000 0000 0000 0000 0000 0000 0000
    ................
    0x00007FC473CAB7A8 : 0000 0000 0000 0000 0000 0000 0000 0000
    ................
    0x00007FC473CAB7B8 : 0000 0000 0000 0000 0000 0000 0000 0000
    ................
    0x00007FC473CAB7C8 : 0000 0000 0000 0000 0100 0000 0000 0000
    ................
    0x00007FC473CAB7D8 : 0100 0000 0000 0000 0200 0000 0000 0000
    ................
    0x00007FC473CAB7E8 : 0000 0000 0000 0000 0000 0000 0000 0000
    ................
    0x00007FC473CAB7F8 : 0000 0000 0000 0000 0000 0000 0000 0000
    ................
    0x00007FC473CAB808 : 0000 0000 0000 0000 0000 0000 0000 0000
    ................
    0x00007FC473CAB818 : 0000 0000 0000 0000 0100 0000 0000 0000
    ................
    0x00007FC473CAB828 : 0100 0000 0000 0000 0100 0000 0000 0000
    ................
    0x00007FC473CAB838 : 0000 0000 0000 0000
    ........
    -- End of gatherStatics --
    curTime: 0
    softMax: 240
    softMaxUnitsTime: true
    alternatePageCleaning: true
    pleaseTerminate: false
    markGBPDep: false
    outOfFileHandlesOccurred: false
    -- Start of stats --
    0x00007FC473CAB858 : 0080 CD73 C47F 0000 0090 CD73 C47F 0000
    ...s.......s....
    0x00007FC473CAB868 : 0A22 1A26 0704 E107 20FE 0500 0000 0000
    .".&.... .......
    0x00007FC473CAB878 : 0000 0000 0000 0000 0000 0000 0000 0000
    ................
    0x00007FC473CAB888 : 0000 0000 0000 0000 9B2E 0100 0000 0000
    ................
    0x00007FC473CAB898 : 0000 0000 0000 0000 0000 0000 0000 0000
    ................
    0x00007FC473CAB8A8 : 0000 0000 0000 0000 0000 0000 0000 0000
    ................
    0x00007FC473CAB8B8 : 0000 0000 0000 0000 0000 0000 0000 0000
    ................
    0x00007FC473CAB8C8 : 0000 0000 0000 0000 0000 0000 0000 0000
    ................
    0x00007FC473CAB8D8 : 0000 0000 0000 0000
    ........
    -- End of stats --
    -- Start of lsnHistoryTable --
    0x00007FC473CAB8E0 : E8F5 0300 0000 0000 0100 0000 0000 0000
    ................
    0x00007FC473CAB8F0 : D102 0000 0000 0000 8F02 0000 0000 0000
    ................
    0x00007FC473CAB900 : D102 0000 0000 0000 A0B0 B3DB C47F 0000
    ................
    -- End of lsnHistoryTable --
    -- Start of initInfo --
    0x00007FC473CAB910 : 00
    .
    -- End of initInfo --
    skippedForSuspend: false
    -- Start of dispatchStats --
    0x00007FC473CAB918 : 0000 0000 0000 0000 0000 0000 0000 0000
    ................
    0x00007FC473CAB928 : 0000 0000 0000 0000 0000 0000 0000 0000
    ................
    0x00007FC473CAB938 : 0000 0000 0000 0000 0000 0000 0000 0000
    ................
    -- End of dispatchStats --
    clnrGlobalIndex: 0
    MESSAGE : AIO failed to collect all outstanding IO requests.
    This is a fatal
              error which must bring down the database.
    CALLSTCK: (Static functions may not be resolved correctly, as
    they are resolved to the nearest symbol)
      [0] 0x00007FC9CEA24B29
    _Z21sqlbClnrCollectAllAIOP12SQLB_CLNR_CB + 0x209
      [1] 0x00007FC9CEAC4689 _Z13sqlbgbCastoutP12SQLB_CLNR_CBb +
    0xF59
      [2] 0x00007FC9CEA16641 _Z19sqlbClnrGatherPagesP12SQLB_CLNR_CB
    + 0x3E1
      [3] 0x00007FC9CEA22198 _Z18sqlbClnrEntryPointP12sqbPgClnrEdu
    + 0x178
      [4] 0x00007FC9CEA21F94 _ZN12sqbPgClnrEdu6RunEDUEv + 0x24
      [5] 0x00007FC9D58EB796 _ZN9sqzEDUObj9EDUDriverEv + 0x116
      [6] 0x00007FC9D3F3E688 sqloEDUEntry + 0x578
      [7] 0x0000003326407A51 /lib64/libpthread.so.0 + 0x7A51
      [8] 0x00000033260E896D clone + 0x6D
    

Local fix

Problem summary

  • ****************************************************************
    * USERS AFFECTED:                                              *
    * DB2 is down when AIO failed to collect all outstanding IO    *
    * requests.                                                    *
    ****************************************************************
    * PROBLEM DESCRIPTION:                                         *
    * See Error Description                                        *
    ****************************************************************
    * RECOMMENDATION:                                              *
    * Upgrade to db2_v111m3fp3                                     *
    ****************************************************************
    

Problem conclusion

Temporary fix

Comments

APAR Information

  • APAR number

    IV95270

  • Reported component name

    DB2 PURESCALE F

  • Reported component ID

    5724Y6900

  • Reported release

    B10

  • Status

    CLOSED FIN

  • PE

    NoPE

  • HIPER

    NoHIPER

  • Special Attention

    NoSpecatt / Xsystem

  • Submitted date

    2017-04-20

  • Closed date

    2018-03-18

  • Last modified date

    2018-03-18

  • APAR is sysrouted FROM one or more of the following:

  • APAR is sysrouted TO one or more of the following:

Fix information

Applicable component levels

  • RB10 PSN

       UP

[{"Business Unit":{"code":"BU053","label":"Cloud & Data Platform"},"Product":{"code":"SSEPGG","label":"Db2 for Linux, UNIX and Windows"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"11.1","Line of Business":{"code":"LOB10","label":"Data and AI"}}]

Document Information

Modified date:
01 August 2020