JR48531: COACHGENERATORCONTEXTMANAGER HAS A MEMORY LEAK THAT LEADS TO A JAVA.LANG.OUTOFMEMORYERROR

Version 8.5 Refresh Pack 5 for the IBM Business Process Manager products
Download Version 8.0.1 Fix Pack 3 for the IBM Business Process Manager products
Version 8.5.0 Fix Pack 2 for the IBM Business Process Manager products

APAR status

• Closed as program error.

Error description

• CoachGeneratorContextManager references a very large
  ConcurrentHashMap. Based on that behavior BPM Server might use
  increased heap memory over time or exhaust the heap leading to
  a java.lang.OutOfMemoryError.
  The heapdump shows that the following.
              |- 1,222,128,976 (40.58%) [32] 19 class
  com/lombardisoftware/server/ejb/workflow/EJSRemoteStatefulEJBWor
  kflowMan
  agerHome_82478d70 0x85e05dc0
                 |- 1,222,128,720 (40.58%) [32] 9 class
  com/lombardisoftware/server/ejb/workflow/EJSStatefulEJBWorkflowM
  anagerHo
  meBean_82478d70 0x85e05d98
                    |- 1,222,128,688 (40.58%) [32] 194 class
  com/lombardisoftware/server/ejb/workflow/EJBWorkflowManagerBean
  0x85e05748
                       |- 1,222,118,864 (40.58%) [32] 66 class
  com/lombardisoftware/component/coach/data/CoachDataBindingHelper
  0x9940fd78
                          |- 1,012,765,248 (33.63%) [32] 13 class
  com/lombardisoftware/component/coachng/gen/CoachGeneratorContext
  Manager
  0x994091e0
                             |- 1,012,765,208 (33.63%) [48] 1
  java/util/concurrent/ConcurrentHashMap 0xa8e961e8
                                |- 1,012,765,160 (33.63%) [80] 16
  array
  of java/util/concurrent/ConcurrentHashMap$Segment 0x98bd03c8
                                   |- 106,112,720 (3.52%) [32] 2
  java/util/concurrent/ConcurrentHashMap$Segment 0x98bd04f0
                                   |- 97,247,528 (3.23%) [32] 2
  java/util/concurrent/ConcurrentHashMap$Segment 0x98bd05d0
                                   |- 95,299,912 (3.16%) [32] 2
  java/util/concurrent/ConcurrentHashMap$Segment 0x98bd04d0
                                   |- 86,491,984 (2.87%) [32] 2
  java/util/concurrent/ConcurrentHashMap$Segment 0x98bd05f0
                                   |- 81,463,240 (2.71%) [32] 2
  java/util/concurrent/ConcurrentHashMap$Segment 0x98bd05b0
                                   |- 81,265,032 (2.7%) [32] 2
  java/util/concurrent/ConcurrentHashMap$Segment 0x98bd0470
                                   |- 69,550,872 (2.31%) [32] 2
  java/util/concurrent/ConcurrentHashMap$Segment 0x98bd0430
                                   |- 66,996,800 (2.22%) [32] 2
  java/util/concurrent/ConcurrentHashMap$Segment 0x98bd0450
                                   |- 63,106,624 (2.1%) [32] 2
  java/util/concurrent/ConcurrentHashMap$Segment 0x98bd0530
                                   |- 62,843,040 (2.09%) [32] 2
  java/util/concurrent/ConcurrentHashMap$Segment 0x98bd0490
                                   |- 58,584,184 (1.95%) [32] 2
  java/util/concurrent/ConcurrentHashMap$Segment 0x98bd0570
                                   |- 58,502,200 (1.94%) [32] 2
  java/util/concurrent/ConcurrentHashMap$Segment 0x98bd0590
                                   |- 47,455,736 (1.58%) [32] 2
  java/util/concurrent/ConcurrentHashMap$Segment 0x98bd0510
                                   |- 37,845,096 (1.26%) [32] 2
  java/util/concurrent/ConcurrentHashMap$Segment 0x98bd0410
                                   |- 56 (0%) [32] 1
  java/util/concurrent/ConcurrentHashMap$Segment 0x98bd04b0
                                   |- 56 (0%) [32] 1
  java/util/concurrent/ConcurrentHashMap$Segment 0x98bd0550
      You may also see HMGR0152W messages.
  [1/30/14 3:10:18:289 EST] 00001449 CoordinatorCo W   HMGR0152W:
  CPU Starvation detected. Current thread scheduling delay is 5
  seconds.
  This due to the memory leak leading to prolonged gc cycles.
  This issue was caused as tip context's were not being cleared
  from the context manager.
  

Local fix

Problem summary

• Playback of coaches on tip are cached causing a memory leak.
  
  EDITIONS AFFECTED:
  IBM Business Process Manager Advanced, Standard and Express
  
  PROBLEM DETAILED DESCRIPTION:
  Playback of coaches should only be cached on deployed snapshots.
  Interim/development Coaches should never be cached, since during
  iterative development cycles many Coaches can be submitted to
  the server which could cause significant memory usage. This
  issue will affect all users in iterative development phase of
  project.
  

Problem conclusion

• A fix for IBM BPM 8.5.0.0 and 8.5.0.1 are available on Fix
  Central: http://www.ibm.com/support/fixcentral. Search for
  JR48531. After you select the product group, product, installed
  version, and platform, click Continue and then select APAR or
  SPR, enter JR48531, and click Continue. When downloading fix
  packages, ensure that you also download the readme file for each
  fix. Review each readme file for additional installation
  instructions and information about the fix.
  
  This fix has been also been delivered as part of fix for
  superseding APAR JR49272, for 8.5.0.1. To obtain, follow the
  instructions above, substituting JR49272 for JR48531.
  

Temporary fix

Comments

APAR Information

• APAR is sysrouted FROM one or more of the following:

• APAR is sysrouted TO one or more of the following:

Fix information

• Fixed component name

  BPM ADVANCED

• Fixed component ID

  5725C9400

Applicable component levels

• R850 PSY

     UP