PH65081: Improve AsyncUpdate processing; Guard against malformed AIProtocolItemStream

APAR status

• Closed as program error.

Error description

• A malformed AIProtocolItemStream, not containing a necessary
  AICompletedPrefixItem, was found persisted in the message store.
  
  When read at startup, this resulted in a malformed AIStream
  object being created with a null _completedPrefix field.
  
  As this field is used for synchronization in the object, this
  subsequently caused a number of NullPointerExceptions to be
  thrown during AsyncUpdate processing.
  
  As this processing was being handled in bundles of
  AsyncUpdates in a pseudo-transactional manner, this caused
  other updates (unconnected except for being scheduled at the
  same time) to be rolled back without any attempt at their
  successful operation.
  
  It was further observed that flaws in the handling of these
  exceptions might also lead to the diminution of the thread
  processing resources for the thread pool used by the
  AsyncUpdate processing, which could lead to slowdown in this
  area of the system.
  Such slowdown was observed.
  

Local fix

• N/A
  

Problem summary

• ****************************************************************
  * USERS AFFECTED:  All users of SIBus Messaging in IBM         *
  *                  WebSphere Application Server                *
  ****************************************************************
  * PROBLEM DESCRIPTION: FFDC NullPointerExceptions arising      *
  *                      from calls to methods on                *
  *                      com.ibm.ws.sib.processor.gd.AIStream;   *
  *                      messaging system slowdown.              *
  ****************************************************************
  * RECOMMENDATION:                                              *
  ****************************************************************
  A malformed AIProtocolItemStream, not containing a necessary
  AICompletedPrefixItem, was found persisted in the message
  store.
  When read at startup, this resulted in a malformed AIStream
  object being created with a null _completedPrefix field.
  As this field is used for synchronization in the object, this
  subsequently caused a number of NullPointerExceptions to be
  thrown during AsyncUpdate processing.
  As this processing was being handled in bundles of
  AsyncUpdates in a pseudo-transactional manner, this caused
  other updates (unconnected except for being scheduled at the
  same time) to be rolled back without any attempt at their
  successful operation.
  It was further observed that flaws in the handling of these
  exceptions might also lead to the diminution of the thread
  processing resources for the thread pool used by the
  AsyncUpdate processing, which could lead to slowdown in this
  area of the system.
  Such slowdown was observed.
  It is likely that the frailties of the AsyncUpdate processing
  led to the malformed AIProtocolItemStream being persisted in
  the first place.
  

Problem conclusion

• Guard code was added to identify, report and discard malformed
  AIProtocolItemStream retrieved from persistence, to avoid the
  creation of malformed AIStream objects.
  
  AsyncUpdate processing was modified to process each update
  within it own (local) transaction, thus avoiding any exception
  in one from causing others to be rolled back without running.
  
  AsyncUpdate processing was enhanced to prevent any
  exception/error thrown from an update from interfering with
  the thread pooling for this processing, thus avoiding the
  possibility of system degradation.
  
  The fix for this APAR is targeted for inclusion in fix packs
  8.5.5.28 and 9.0.5.24.
  For more information, see 'Recommended Updates for
  WebSphere Application Server':
  https://www.ibm.com/support/pages/node/715553
  

Temporary fix

Comments

APAR Information

• APAR is sysrouted FROM one or more of the following:

• APAR is sysrouted TO one or more of the following:

Fix information

• Fixed component name

  WEBS APP SERV N

• Fixed component ID

  5724H8800

Applicable component levels