PH26976: With XM enabled, customer may see elevated CPU usage when subscribe/resubscribe is called after a disconnection.

APAR status

• Closed as program error.

Error description

• When extreme memory is enabled , and clients try to
  subscribe/resubscribe after a disconnections , some of the
  containers may experience elevated CPU usage.
  

Local fix

Problem summary

• ****************************************************************
  * USERS AFFECTED:  Users of eXtreme Scale who use near-cache   *
  *                  and eXtremeMemory.                          *
  ****************************************************************
  * PROBLEM DESCRIPTION: Clients which are trying to             *
  *                      subscribe/resubscribe                   *
  *                      after a disconnections are waiting      *
  *                      for lock.                               *
  ****************************************************************
  * RECOMMENDATION:                                              *
  ****************************************************************
  The
  SubscriptionProcessPool thread which has locked
  subscriptionReceivedManagedSubscriberList for long time.
  The stack trace for the SubscriptionProcessPool
  thread which has
  locked subscriptionReceivedManagedSubscriberList for long time
  is as
  following :
  3XMTHREADINFO "SubscriptionProcessPool : 98"
  J9VMThread:0x0000000007E65E00, omrthread_t:0x00007FB9D8038BC0,
  java/lang/Thread:0x000000005613BAA8, state:R, prio=5
  3XMJAVALTHREAD (java/lang/Thread getId:0x15DA, isDaemon:true)
  3XMTHREADINFO1 (native thread ID:0x3CC7, native priority:0x5,
  native policy:UNKNOWN, vmstate:CW, vm thread
  flags:0x00000081)
  3XMTHREADINFO2 (native stack address range
  from:0x00007FB948106000, to:0x00007FB948147000, size:0x41000)
  3XMCPUTIME CPU usage total: 16.623661185 secs, current
  category="Application"
  3XMHEAPALLOC Heap bytes allocated
  since last GC cycle=1056576 (0x101F40)
  3XMTHREADINFO3 Java
  callstack:
  4XESTACKTRACE at com/ibm/ws/objectgrid/io/offheap/i
  mpl/XsOffHeapMapImpl.getXsValue_ByteArray(Native Method)
  4XESTACKTRACE at
  com/ibm/ws/objectgrid/io/offheap/impl/XsOffHeap
  MapImpl.getAndPin(XsOffHeapMapImpl.java:368(Compiled Code))
  4XESTACKTRACE at
  com/ibm/ws/objectgrid/io/offheap/ObjectGridHash
  TableOH.getInternal(ObjectGridHashTableOH.java:880(Compiled
  Code))
  4XESTACKTRACE at com/ibm/ws/objectgrid/io/offheap/Objec
  tGridHashTableOH.getAndPin(ObjectGridHashTableOH.java:801(Compil
  ed Code))
  4XESTACKTRACE at com/ibm/ws/objectgrid/io/offheap/Ob
  jectGridHashTableOH.nextInternal(ObjectGridHashTableOH.java:1162
  (Compiled Code))
  4XESTACKTRACE at com/ibm/ws/objectgrid/io/off
  heap/ObjectGridHashTableOH.next(ObjectGridHashTableOH.java:1115(
  Compiled Code))
  4XESTACKTRACE at com/ibm/ws/objectgrid/io/offh
  eap/ObjectGridHashTableOH.next(ObjectGridHashTableOH.java:86(Com
  piled Code))
  4XESTACKTRACE at com/ibm/ws/objectgrid/util/Objec
  tGridHashtableA.getChangedSet(ObjectGridHashtableA.java:434(Comp
  iled Code))
  4XESTACKTRACE at com/ibm/ws/objectgrid/map/BaseMap
  .getChangeSet(BaseMap.java:12032(Compiled Code))
  4XESTACKTRACE at
  com/ibm/ws/xs/cacheinvalidator/server/NearCache
  InvalidationPublisher.subscriptionReceivedManagedSubscriberList(
  NearCacheInvalidationPublisher.java:195(Compiled Code))
  4XESTACKTRACE at
  com/ibm/ws/xs/pubsub/publication/Publisher.subs
  cribe(Publisher.java:436(Compiled Code))
  5XESTACKTRACE
  (entered lock: java/util/LinkedList@0x000000004E3B78C0, entry
  count: 1)
  4XESTACKTRACE at com/ibm/ws/xs/pubsub/publication/Pu
  blisher$SubscriptionRequestThread.run(Publisher.java:872(Compile
  d Code))
  4XESTACKTRACE at java/util/concurrent/ThreadPoolExecu
  tor.runWorker(ThreadPoolExecutor.java:1160(Compiled Code))
  4XESTACKTRACE at
  java/util/concurrent/ThreadPoolExecutor$Worker.
  run(ThreadPoolExecutor.java:635(Compiled Code))
  4XESTACKTRACE
  at
  com/ibm/ws/objectgrid/thread/XSThreadPool$Worker.run(XSThread
  Pool.java:309(Compiled Code))
  

Problem conclusion

• WXS code has been changed to avoid unnecessary locking. An
  interim fix is available for this APAR upon request.
  

Temporary fix

Comments

APAR Information

• APAR is sysrouted FROM one or more of the following:

• APAR is sysrouted TO one or more of the following:

Fix information

• Fixed component name

  WS EXTREME SCAL

• Fixed component ID

  5724X6702

Applicable component levels