PH01832: High CPU observed on the dmgr process driven by VisualizationEngine.CacheWorkItemsTP thread.

9.0.0.10: WebSphere Application Server traditional V9.0 Fix Pack 10
8.5.5.15: WebSphere Application Server V8.5.5 Fix Pack 15
9.0.0.11: WebSphere Application Server traditional V9.0 Fix Pack 11
9.0.5.0: WebSphere Application Server traditional Version 9.0.5 Refresh Pack
9.0.5.1: WebSphere Application Server traditional Version 9.0.5 Fix Pack 1
9.0.5.2: WebSphere Application Server traditional Version 9.0.5 Fix Pack 2
8.5.5.17: WebSphere Application Server V8.5.5 Fix Pack 17
9.0.5.3: WebSphere Application Server traditional Version 9.0.5 Fix Pack 3
8.5.5.20: WebSphere Application Server V8.5.5.20
8.5.5.18: WebSphere Application Server V8.5.5 Fix Pack 18
8.5.5.19: WebSphere Application Server V8.5.5 Fix Pack 19
8.5.5.16: WebSphere Application Server V8.5.5 Fix Pack 16
8.5.5.21: WebSphere Application Server V8.5.5.21

APAR status

• Closed as program error.

Error description

• High CPU observed on the dmgr process driven by
  VisualizationEngine.CacheWorkItemsTP thread and the
  javacores show entries similar to:
  
  "VisualizationEngine.CacheWorkItemsTP : 21"
  J9VMThread:0x0000000005C2E500,
  j9thread_t:0x00007F30F8042D80,
  java/lang/Thread:0x00000007072ABD30, state:R, prio=5
  (java/lang/Thread getId:0x2CD, isDaemon:true)
  (native thread ID:0x89AB, native priority:0x5, native
  policy:UNKNOWN, vmstate:CW, vm thread flags:0x00000001)
  (native stack address range from:0x00007F3152335000,
  to:0x00007F3152376000, size:0x41000)
  CPU usage total: 41682.791755135 secs
  Heap bytes allocated since last GC cycle=0 (0x0)
  Java callstack:
  at java/util/HashMap.getEntry(HashMap.java:477(Compiled
  Code))
  at java/util/HashMap.containsKey(HashMap.java:462(Compiled
  Code))
  at
  com/ibm/ws/xd/visualizationengine/cacheservice/cacheimpl/Ser
  verStatsCache$ServerStatsAsyncClient.start(ServerStatsCache.
  java:1460)
  at
  com/ibm/ws/xd/visualizationengine/cacheservice/cacheimpl/Ser
  verStatsCache.initializeServer(ServerStatsCache.java:629)
  at
  com/ibm/ws/xd/visualizationengine/cacheservice/cacheimpl/Ser
  verStatsCache.initializeEntityCache(ServerStatsCache.java:53
  7)
  at
  com/ibm/ws/xd/visualizationengine/cacheservice/AbstractIndex
  Cache$InitializeEntityWorker.run(AbstractIndexCache.java:146
  5)
  at
  com/ibm/ws/util/ThreadPool$Worker.run(ThreadPool.java:1892(C
  ompiled Code))
  

Local fix

• N/A
  

Problem summary

• ****************************************************************
  * USERS AFFECTED:  All users of IBM WebSphere Application      *
  *                  Server WAS ND edition- Virtual              *
  *                  Enterprise/IM Component                     *
  ****************************************************************
  * PROBLEM DESCRIPTION: High CPU observed on the dmgr process   *
  *                      driven by                               *
  *                      VisualizationEngine.CacheWorkItemsTP    *
  *                      thread.                                 *
  ****************************************************************
  * RECOMMENDATION:                                              *
  ****************************************************************
  High CPU observed on the dmgr process driven by
  VisualizationEngine.CacheWorkItemsTP thread and the javacores
  show entries similar to:
  
  "VisualizationEngine.CacheWorkItemsTP : 21"
  J9VMThread:0x0000000005C2E500, j9thread_t:0x00007F30F8042D80,
  java/lang/Thread:0x00000007072ABD30, state:R, prio=5
  (java/lang/Thread getId:0x2CD, isDaemon:true)
  (native thread ID:0x89AB, native priority:0x5, native
  policy:UNKNOWN, vmstate:CW, vm thread flags:0x00000001)
  (native stack address range from:0x00007F3152335000,
  to:0x00007F3152376000, size:0x41000)
  CPU usage total: 41682.791755135 secs
  Heap bytes allocated since last GC cycle=0 (0x0)
  Java callstack:
  at java/util/HashMap.getEntry(HashMap.java:477(Compiled Code))
  at java/util/HashMap.containsKey(HashMap.java:462(Compiled
  Code))
  at
  com/ibm/ws/xd/visualizationengine/cacheservice/cacheimpl/ServerS
  tatsCache$ServerStatsAsyncClient.start(ServerStatsCache.java:146
  0)
  at
  com/ibm/ws/xd/visualizationengine/cacheservice/cacheimpl/ServerS
  tatsCache.initializeServer(ServerStatsCache.java:629)
  at
  com/ibm/ws/xd/visualizationengine/cacheservice/cacheimpl/ServerS
  tatsCache.initializeEntityCache(ServerStatsCache.java:537)
  at
  com/ibm/ws/xd/visualizationengine/cacheservice/AbstractIndexCach
  e$InitializeEntityWorker.run(AbstractIndexCache.java:1465)
  at
  com/ibm/ws/util/ThreadPool$Worker.run(ThreadPool.java:1892(Compi
  led Code))
  

Problem conclusion

• The synchronization around the HashMap referenced in the stack
  was incorrect and allowed the map to grow unbounded resulting in
  the high cpu condition on the dmgr. The
  visualization.cacheservice component has been modified to
  properly guard the HashMap object to avoid the high cpou
  condition.
  
  The fix for this APAR is currently targeted for inclusion in fix
  packs 9.0.0.9 and 8.5.5.15.  Please refer to the Recommended
  Updates page for delivery information:
  http://www.ibm.com/support/docview.wss?rs=180&uid=swg27004980
  

Temporary fix

Comments

APAR Information

• APAR is sysrouted FROM one or more of the following:

• APAR is sysrouted TO one or more of the following:

Fix information

• Fixed component name

  WEBS APP SERV N

• Fixed component ID

  5724H8800

Applicable component levels

• R850 PSY

     UP