IBM Support

IT23954: HADR STANDBY FAILS WITH DBHEAP OUT OF MEMORY OR MEMORY CORRUPTION

Subscribe

You can track all active APARs for this component.

 

APAR status

  • Closed as program error.

Error description

  • The memory assigned to DB2_HADR_BUF_SIZE db2 registry is being
    allocated from dbheap during database activation/connection.
    Any changed to this registry become effective after db2 restart.
    
    This defect can cause a) unnecessary re-allocation of the
    receive
    buffer leading to the Standby running out of memory or b) memory
    corruption due to not allocating memory for the receive buffer
    when
    necessary.
    
    Symptom a) DBHEAP OUT OF MEMORY
    
    This defect causes re-allocation of DB2_HADR_BUF_SIZE memory
    during standby db re-connect after communication failure when
    there is no change to the setting. This causes double allocation
    of memory from dbheap.
    
    And since some customers set the value of this DB2_HADR_BUF_SIZE
    registry to a higher number than the default of 2Xlogbufsz, it
    may cause out of dbheap memory errors on Standby causing db2 to
    go down.
    
    the messages in db2diag.log may look like this:
    
    2018-01-21-01.31.15.322975-300 E12975694A696        LEVEL:
    Warning
    PID     : 18612304             TID : 20562          PROC :
    db2sysc 0
    INSTANCE: db2inst1             NODE : 000           DB   :
    SAMPLE
    HOSTNAME: standbyhost
    EDUID   : 20562                EDUNAME: db2hadrs.0.0 (SAMPLE) 0
    FUNCTION: DB2 UDB, SQO Memory Management,
    sqloMemLogPoolConditions, probe:30
    DATA #1 : <preformatted>
    Out of memory failure for Database Heap (DBHEAP) on node 0.
    Requested block size &#160; &#160; &#160; &#160; &#160; : 4295032808 bytes.
    Physical heap size &#160; &#160; &#160; &#160; &#160; &#160; : 5139988480 bytes.
    Configured heap size &#160; &#160; &#160; &#160; &#160; : 9320792064 bytes.
    Unreserved memory used by heap : 0 bytes.
    Unreserved memory left in set &#160;: 470089728 bytes.
    
    This is most likely to occur when using a very large
    DB2_HADR_BUF_SIZE up to the 4GB recommended limit, in
    conjunction with low db memory.
    
    With this fix, we will not re-allocate memory if no change to
    the setting.
    
    Symptom b) MEMORY CORRUPTION
    
    Under the following conditions:
    
    1. logbufsz on the Primary database is exactly 32768
    
    2. DB2_HADR_BUF_SIZE is unset on the Standby
    
    memory corruption occurs and the Standby database
    crashes with an FODC_Trap or FODC_Panic as a result.
    

Local fix

  • Symptom a) DBHEAP OUT OF MEMORY
    
    As a temporary workaround, please apply one of the following:
    
    1. Increase database memory
    
    or
    
    2. Use log spooling instead of using large DB2_HADR_BUF_SIZE
    
    Symptom b) MEMORY CORRUPTION
    
    Set the DB2_HADR_BUF_SIZE registry variable on the Standby to
    any value other than the default of 16, e.g.
    
    db2set DB2_HADR_BUF_SIZE=65536
    
    The Standby receive buffer will be correctly allocated when the
    Standby is brought back up.
    

Problem summary

  • ****************************************************************
    * USERS AFFECTED:                                              *
    * ALL                                                          *
    ****************************************************************
    * PROBLEM DESCRIPTION:                                         *
    * See Error Description                                        *
    ****************************************************************
    * RECOMMENDATION:                                              *
    * The complete fix for this problem first appears in DB2       *
    * Version 11.1.3.3 iFix001 and all the subsequent Fix Packs.   *
    ****************************************************************
    

Problem conclusion

  • The complete fix for this problem first appears in DB2 Version
    11.1.3.3 iFix001 and all the subsequent Fix Packs.
    

Temporary fix

Comments

APAR Information

  • APAR number

    IT23954

  • Reported component name

    DB2 FOR LUW

  • Reported component ID

    DB2FORLUW

  • Reported release

    B10

  • Status

    CLOSED PER

  • PE

    NoPE

  • HIPER

    NoHIPER

  • Special Attention

    NoSpecatt / Xsystem

  • Submitted date

    2018-02-05

  • Closed date

    2018-05-22

  • Last modified date

    2019-04-30

  • APAR is sysrouted FROM one or more of the following:

  • APAR is sysrouted TO one or more of the following:

    IT23988

Fix information

  • Fixed component name

    DB2 FOR LUW

  • Fixed component ID

    DB2FORLUW

Applicable component levels

  • RB10 PSN

       UP

[{"Business Unit":{"code":"BU053","label":"Cloud & Data Platform"},"Product":{"code":"SSEPGG","label":"Db2 for Linux, UNIX and Windows"},"Component":"","ARM Category":[],"Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"11.1","Edition":"","Line of Business":{"code":"LOB10","label":"Data and AI"}}]

Document Information

Modified date:
30 April 2019