IT35216: OOM: REQUEST MEMORY IS FAILING UNNECESSARILY EARLY WHEN THERE ISCONTENTION TRYING TO STEAL MEMORY FROM OTHER CONSUMERS

APAR status

• Closed as program error.

Error description

• DB2 may report out of memory (OOM) errors even if there is
  sufficient cached database memory available.
  OOMs could probably have been avoided altogether if the request
  for Instance Memory (requestMemory) tried a bit harder to
  reclaim cached memory in the other consumers.
  
  For example, db2diag.log shows:
  
  2017-07-17-10.12.13.088021+000 I929503368E1120       LEVEL:
  Warning
  PID     : 10598                TID : 46970232104704  PROC :
  db2sysc 21
  INSTANCE: db2inst1             NODE : 021            DB   :
  BLUDB
  APPHDL  : 0-27126              APPID:
  10.113.34.46.62106.170717100710
  AUTHID  : ELTUSER              HOSTNAME: host
  EDUID   : 369804               EDUNAME: db2agntcol (BLUDB) 21
  FUNCTION: DB2 UDB, SQO Memory Management,
  SqloMemController::requestMemory, probe:50
  MESSAGE : ZRC=0x8B0F0000=-1961951232=SQLO_NOMEM "No Memory
  Available"
            DIA8300C A memory heap error has occurred.
  DATA #1 : String, 36 bytes
  OOM - Instance memory request failed
  DATA #2 : String, 35 bytes
  Logging disabled until next success
  DATA #3 : unsigned integer, 8 bytes
  34406400
  DATA #4 : unsigned integer, 8 bytes
  10
  DATA #5 : String, 13 bytes
  APPL-BLUDB
  DATA #6 : unsigned integer, 8 bytes
  1439105024
  DATA #7 : unsigned integer, 8 bytes
  0
  DATA #8 : unsigned integer, 8 bytes
  32124362752
  DATA #9 : unsigned integer, 8 bytes
  32131727360
  DATA #10: unsigned integer, 8 bytes
  3226992640
  
  ...
  APPL-BLUDB    - Current size : 1405376 KB, HWM : 2827136 KB,
  Cached : 0 KB
  DBMS-db2inst1 - Current size : 156096 KB, HWM : 162880 KB,
  Cached : 0 KB
  FMP_RESOURCES - Current size : 768 KB, HWM : 22528 KB, Cached :
  0 KB
  PRIVATE - Current size : 167040 KB, HWM : 340608 KB, Cached : 0
  KB
  FCM_RESOURCES - Current size : 104312 KB, HWM : 197376 KB,
  Cached : 0 KB
  DB-BLUDB    - Current size : 29537856 KB, HWM : 30081408 KB,
  Cached : 3184960 KB
  
  
  The typical db2diag.log entry will show "10" for the
  retryCounter value, which is DATA#4:
  The request size is DATA #3, 34MB
  APPL memory size is DATA #6, 1.4GB, not unreasonable.  DATA #7
  is its cached amount, 0.
  Limit is DATA #9, 32GB, DATA #8 shows usage we are at the limit
  But DATA #10 shows 3.2GB Cached, plenty to satisfy a 32MB
  request, and from subsequent entries we could see it´s all in
  Database Memory.
  So we can conclude we were hitting contention trying to reduce
  (the cached/overflow amount) in database memory set.
  It must be busy due to sort memory growth and/or decrease across
  the various sort pools.
  

Local fix

Problem summary

• ****************************************************************
  * USERS AFFECTED:                                              *
  * ALL                                                          *
  ****************************************************************
  * PROBLEM DESCRIPTION:                                         *
  * See Error Description                                        *
  ****************************************************************
  * RECOMMENDATION:                                              *
  * Upgrade to Db2 Version 11.1 Mod 4 Fix Pack 6.                *
  ****************************************************************
  

Problem conclusion

• First fixed in Db2 Version 11.1 Mod 4 Fix Pack 6.
  

Temporary fix

Comments

APAR Information

• APAR is sysrouted FROM one or more of the following:

• APAR is sysrouted TO one or more of the following:

Fix information

• Fixed component name

  DB2 FOR LUW

• Fixed component ID

  DB2FORLUW

Applicable component levels