Memory leak in WAS 8.5.x J2C PoolManager

Content

A memory leak in the J2C PoolManager can potentially lead to an OutOfMemory (OOM) condition.

Symptoms may include high CPU usage and the following FFDC:
FFDC Exception:java.lang.Exception
SourceId:com.ibm.ejs.j2c.PoolManager$2 ProbeId:50 Reporter:com.ibm.ejs.j2c.PoolManager$@ab1234ef
java.lang.Exception: WSThreadLocal: instance count = 200 ; potential memory leak; verify usage.

Heapanalyzer might show a leak suspect with a stack trace similar to this:

A Linked List Data Structure Detected.                              
1,981,662,352 bytes (47.28 %) of Java heap is used by 44 instances of   java/util/concurrent/ConcurrentSkipListMap$Index                
Contains the following object:- 44 instances of java/util/concurrent/ConcurrentSkipListMap$Node holding 5,366,608 bytes                                          
                                                                     
TotalSize [ObjectSize] NumberOfChildObject(249,874) ObjectName Address    
|- 1,981,966,928 [24] 1 java/util/concurrent/ConcurrentSkipListMap$HeadIndex 0xf7766298                                                
   |- 1,981,966,904 [24] 2 java/util/concurrent/ConcurrentSkipListMap$HeadIndex 0xf180b1b8                                                
      |- 1,981,966,544 [24] 3 java/util/concurrent/ConcurrentSkipListMap$Index 0xffa75200                                                
         |- 1,453,166,232 [24] 3 java/util/concurrent/ConcurrentSkipListMap$Index 0xffa751e8                      
         |- 528,679,664 [24] 3 java/util/concurrent/ConcurrentSkipListMap$Index 0x1055a3258                      
         |- 120,624 [24] 3 java/util/concurrent/ConcurrentSkipListMap

This memory leak specifically manifests itself when applications use the JMS Queue Connection Factory where each JMS Managed Connection has an associated JMS Connection pool and JMS Session pool. When the Managed connection is destroyed due to unused or aged timeouts or the connection is stale, then the associated JMS Session pool will be stopped/destroyed and the reaper alarms associated with the PoolManager should also be cancelled.

The Session pool is being stopped, however, the PoolManager instances registered in the alarms never get destroyed and the alarms are repeatedly created and cancelled for every reap cycle. This causes the PoolManager objects and its associated reaper alarm objects to stay on the heap forever, potentially leading to OOM conditions.

This is an example of an application using a Queue Connection Factory, making it susceptible to this issue:

This memory leak problem is addressed by APAR PI14746 which is targetted for inclusion in WebSphere Application Server fixpack v8.5.5.3.

An ifix for level 8.5.5.0, 8.5.5.1, and 8.5.5.2 can be downloaded from FixCentral at

http://www-01.ibm.com/support/docview.wss?uid=swg24037572

Note: If you have other ifixes installed, please contact support to make sure there is no conflict.