IBM Support

IT18873: MEMORY LEAK IN CDE-SORT WHEN USING BLU-MPP ENVIRONMENT

Subscribe

You can track all active APARs for this component.

 

APAR status

  • Closed as program error.

Error description

  • When running a query which requires a sort on a BLU-MPP
    environment, a memory leak could be hit which could later result
    in an out-of-memory condition.
    The out-of-memory condition is hit with the CDE-MERGE evaluator
    when merging sorted results from different members of the MPP
    cluster.
    
    Following entries will be logged into the db2diag.log
    
    2016-12-21-11.44.12.840285+540 I98635417E908         LEVEL:
    Severe
    PID     : 9345                 TID : 140724683138816 PROC :
    db2sysc 0
    INSTANCE: db2inst1             NODE : 000            DB   :
    DBNAME
    APPHDL  : 0-3992               APPID: *N0.db2inst1.000000000000
    AUTHID  : DB2                  HOSTNAME:
    hostname
    EDUID   : 292                  EDUNAME: db2agntcol (DBNAME) 0
    FUNCTION: DB2 UDB, CDE sort, MergeInfo::throwOutOfMemory,
    probe:967
    DATA #1 : String, 237 bytes
    SORTHEAP needs to be larger.
     MergeEvaluator.cpp 401
     void ibm_cde::query::MergeEvaluator::processWorkunits()
     Required Surplus: 640 bytes
     Current surplus: -992 bytes
     InUse:           2147484640 bytes
     Reserved:        2147483648 bytes
    DATA #2 : Function name in Library, 0 bytes
    Object not dumped: Address: 0x0000000000000000 Size: 0 Reason:
    Address
    is NULL
    DATA #3 : File name, 13 bytes
    MergeInfo.cpp
    
    2016-12-21-11.44.13.485354+540 I98636326E892         LEVEL:
    Severe
    PID     : 9345                 TID : 140724683138816 PROC :
    db2sysc 0
    INSTANCE: db2inst1             NODE : 000            DB   :
    DBNAME
    APPHDL  : 0-3992               APPID: *N0.db2inst1.000000000000
    AUTHID  : DB2                  HOSTNAME:
    hostname
    EDUID   : 292                  EDUNAME: db2agntcol (DBNAME) 0
    FUNCTION: DB2 UDB, CDE Services,
    ibm_cde::services::MemConsumer::throwOutOfHeapMemoryError,
    probe:551
    DATA #1 : String, 189 bytes
    Unable to reserve more heap memory.
     Consumer:        7
     Memory limit:    2147483648 bytes
     Reserved memory: 2147483648 bytes
     Memory in use:   2147484640 bytes
     Surplus:         -992 bytes
    DATA #2 : Function name in Library, 0 bytes
    Object not dumped: Address: 0x0000000000000000 Size: 0 Reason:
    Address
    is NULL
    DATA #3 : File name, 11 bytes
    Subpool.cpp
    
    2016-12-21-11.44.14.272407+540 I98637219E3031        LEVEL:
    Error
    PID     : 9345                 TID : 140724683138816 PROC :
    db2sysc 0
    INSTANCE: db2inst1             NODE : 000            DB   :
    DBNAME
    APPHDL  : 0-3992               APPID: *N0.db2inst1.000000000000
    AUTHID  : DB2                  HOSTNAME:
    hostname
    EDUID   : 292                  EDUNAME: db2agntcol (DBNAME) 0
    FUNCTION: DB2 UDB, CDE Trace, cdeDiagnosticsFacility, probe:560
    DATA #1 : String, 332 bytes
    DB2ErrorException: DB2 Error Exception zRC=0x0, SQLCODE=-955,
    diagnostics:
    SQLS_MEMCONSUMER_CDE_MERGE stats (4K pages)
    current state:
     shsort:
    sortheap: 524288
    sheapthres: 5452592
    reserved, HWM: 2011523, 2011523
    physical usage: 2332816
     consumer:
    reserved: 524288
    logical usage: 524288
    Thrown at Subpool.cpp : 560
    DATA #2 : Function name in Library, 68 bytes
    void
    ibm_cde::services::MemConsumer::throwOutOfHeapMemoryError(bool)
    DATA #3 : File name, 11 bytes
    Subpool.cpp
    CALLSTCK: (Static functions may not be resolved correctly, as
    they are
    resolved to the nearest symbol)
      [0] 0x00007FFD1B3D18A9
    _ZN7ibm_cde8services6Tracer9traceDataENS0_11ComponentIDENS0_16Co
    mmonTrac
    eLevelENS1_10TraceEventEPKcmS6_PKvmRKSbIcSt11char_trait + 0x369
      [1] 0x00007FFD1B372A32
    _ZNK7ibm_cde8services13BaseException14traceExceptionENS0_11Compo
    nentIDEP
    Kc + 0x322
      [2] 0x00007FFD1B39CDD1
    _ZN7ibm_cde8services11MemConsumer25throwOutOfHeapMemoryErrorEb +
    0x291
      [3] 0x00007FFD1DD0510D
    _ZN7ibm_cde5query9MergeInfo16throwOutOfMemoryEPKcmS3_l + 0x10D
      [4] 0x00007FFD1DD0AB95
    _ZN7ibm_cde5query9MergeInfo16reserveToSurplusEPKcmS3_l + 0x225
      [5] 0x00007FFD1DD86718
    _ZN7ibm_cde5query14MergeEvaluator16processWorkunitsEv + 0x3F8
      [6] 0x00007FFD1DD850FA
    _ZN7ibm_cde5query14MergeEvaluator26processInputsSynchronouslyEv
    + 0xAA
      [7] 0x00007FFD1C36B729
    _ZN7ibm_cde5query9Evaluator8evaluateEbbRNS1_21EvaluatorRestartSt
    ateEPNS0
    _19OptPredicateTrackerE + 0x3A9
      [8] 0x00007FFD1C28948C
    _ZN7ibm_cde5query17EvaluationRoutine8evaluateEjP15sql_static_dat
    a +
    0x30C
      [9] 0x00007FFD1CCB8B35
    _ZN7ibm_cde5query9Scheduler13evaluateChainEPNS0_17EvaluationRout
    ineERm +
    0x285
      [10] 0x00007FFD1CCBC013
    _ZN7ibm_cde5query9Scheduler15runWorkerThreadEPvPi + 0x363
      [11] 0x00007FFD1CCCA99E
    _ZN7ibm_cde5query17cdeEntryPointImplEP8sqeAgentPvS3_ + 0x6E
      [12] 0x00007FFD1D8CC02E
    _ZN12cdeInterface16startCdeSubagentEP8sqeAgent
    + 0x27E
      [13] 0x00007FFD21033FCF _Z14sqlriInvokeCdeP8sqlrr_cb + 0x3F
      [14] 0x00007FFD20E96BF8
    _Z15sqlriSectInvokeP8sqlrr_cbP12sqlri_opparm +
    0x4F8
      [15] 0x00007FFD208FC067
    _Z21sqlrr_subagent_routerP8sqeAgentP12SQLE_DB2RA_T + 0xD87
      [16] 0x00007FFD1E4907DD
    /db2/db2inst1/sqllib/lib64/libdb2e.so.1 +
    0x532B7DD
      [17] 0x00007FFD1E48F0CA _Z21sqleProcessSubRequestP8sqeAgent +
    0x3DA
      [18] 0x00007FFD1E4C28CB _ZN8sqeAgent6RunEDUEv + 0x5AB
      [19] 0x00007FFD21C7F796 _ZN9sqzEDUObj9EDUDriverEv + 0x116
      [20] 0x00007FFD202D2688 sqloEDUEntry + 0x578
      [21] 0x00007FFD2741BDF5 /lib64/libpthread.so.0 + 0x7DF5
      [22] 0x00007FFD17C3C1AD clone + 0x6D
    

Local fix

Problem summary

  • ****************************************************************
    * USERS AFFECTED:                                              *
    * ALL                                                          *
    ****************************************************************
    * PROBLEM DESCRIPTION:                                         *
    * See Error Description                                        *
    ****************************************************************
    * RECOMMENDATION:                                              *
    * Upgrade to DB2 11.1 Mod 2 Fix Pack 2 or higher               *
    ****************************************************************
    

Problem conclusion

  • First fixed in DB2 11.1 Mod 2 Fix Pack 2
    

Temporary fix

Comments

APAR Information

  • APAR number

    IT18873

  • Reported component name

    DB2 FOR LUW

  • Reported component ID

    DB2FORLUW

  • Reported release

    B10

  • Status

    CLOSED PER

  • PE

    NoPE

  • HIPER

    NoHIPER

  • Special Attention

    NoSpecatt / Xsystem

  • Submitted date

    2017-01-17

  • Closed date

    2017-06-23

  • Last modified date

    2017-06-23

  • APAR is sysrouted FROM one or more of the following:

  • APAR is sysrouted TO one or more of the following:

Fix information

  • Fixed component name

    DB2 FOR LUW

  • Fixed component ID

    DB2FORLUW

Applicable component levels

[{"Business Unit":{"code":"BU053","label":"Cloud & Data Platform"},"Product":{"code":"SSEPGG","label":"DB2 for Linux- UNIX and Windows"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"11.1","Line of Business":{"code":"LOB10","label":"Data and AI"}}]

Document Information

Modified date:
29 June 2020