A fix is available
APAR status
Closed as program error.
Error description
A hang condition may occur on an application thread that is using the MQ V7.5 classes for JMS application and attempting to close a JMS MessageConsumer object, which is associated with a JMS MessageListener for asynchronous message delivery, at the same time as the MQ classes for JMS receive a notification from the queue manager that is shutting down or quiescing. An FFDC file is generated reporting a java.lang.NullPointerException within the MQ classes for JMS code: Source Class :-com.ibm.mq.jmqi.remote.impl.RemoteDispatchThread Source Method :- run() ProbeID :- 01 Thread :- name=DispatchThread: [com.ibm.mq.jmqi.remote.impl.RemoteSession[ connectionId=414D5143444D4F4D5445535430332020D29C045801270020]] priority=5 group=JMSCCThreadPoolccl=sun.misc.Launcher$AppClassLoader@18b4aa c2 Data ---- | Description :- Unexpected error | Exception Summary :- java.lang.NullPointerException[null] | Exception message :- <null> Stack trace ----------- Stack trace to show the location of the FFST call | FFST Location :- java.lang.Exception | at com.ibm.msg.client.commonservices.trace.Trace.getCurrentPosition (Trace.java:2019) | at com.ibm.msg.client.commonservices.trace.Trace.createFFSTString(T race.java:1958) | at com.ibm.msg.client.commonservices.trace.Trace.ffstInternal(Trace .java:1847) | at com.ibm.msg.client.commonservices.trace.Trace.ffst(Trace.java:16 60) | at com.ibm.mq.jmqi.remote.impl.RemoteDispatchThread.run(RemoteDispa tchThread.java:391) | at com.ibm.msg.client.commonservices.workqueue.WorkQueueItem.runTas k(WorkQueueItem.java:214) A Javacore (thread dump) showing threads within the JVM will show the hung application thread that is in an API call to MQQueueReceiver.close() with the following Java call stack: "THREADNAME" daemon prio=5 os_prio=0 tid=0x00007fa0dc003000 nid=0x33fe in Object.wait() java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Object.java:502) at com.ibm.mq.jmqi.remote.api.RemoteHconn.waitOnDispatchEvent(Remot eHconn.java:502) - locked <0x00000000feecdb88> (a com.ibm.mq.jmqi.remote.api.RemoteHconn$DispatchEventSync) at com.ibm.mq.jmqi.remote.api.RemoteHconn.dispatchThreadExchange(Re moteHconn.java:545) at com.ibm.mq.jmqi.remote.api.RemoteFAP.MQCTL(RemoteFAP.java:2562) at com.ibm.mq.jmqi.monitoring.JmqiInterceptAdapter.MQCTL(JmqiInterc eptAdapter.java:216) at com.ibm.msg.client.wmq.internal.WMQConsumerOwnerShadow.suspendAs yncService(WMQConsumerOwnerShadow.java:453) at com.ibm.msg.client.wmq.internal.WMQSession.suspendAsyncService(W MQSession.java:1499) at com.ibm.msg.client.wmq.internal.WMQConsumerShadow.close(WMQConsu merShadow.java:326) - locked <0x00000000feefadb8> (a java.lang.Object) at com.ibm.msg.client.wmq.internal.WMQMessageConsumer.close(WMQMess ageConsumer.java:378) at com.ibm.msg.client.jms.internal.JmsMessageConsumerImpl.close(Jms MessageConsumerImpl.java:266) - locked <0x00000000feefad18> (a com.ibm.msg.client.jms.internal.State) at com.ibm.msg.client.jms.internal.JmsMessageConsumerImpl.close(Jms MessageConsumerImpl.java:185) at com.ibm.mq.jms.MQQueueReceiver.close(MQQueueReceiver.java:84)
Local fix
Three local workarounds are available: Option 1 ---------- Use a transacted JMS Session within the application to create the JMS QueueReceiver objects that are then used with the JMS MessageListeners. For example, rather than calling: JmsQueueConnectionImpl.createQueueSession( false, Session.AUTO_ACKNOWLEDGE) for a non-transacted JMS Session, pass the boolean value: true as the first argument to ensure a transacted JMS Session is used as messages that are delivered to the application's JMS MessageListener are consumed under an MQ syncpoint. Option 2 ---------- Use the IBM MQ V8.0.0.5 (or later) classes for JMS. Some of the MQ V8.0.0.5 classes for JMS code was refactored in this version under APAR IT11291 which prevents the java.lang.NullPointerException occurring, which leads to the hung application thread performing the JMS MessageConsumer.close() call. Option 3 ---------- Kill and restart the application.
Problem summary
**************************************************************** USERS AFFECTED: This issue affects users of the WebSphere MQ V7.5 classes for JMS who are consuming messages asynchronously using JMS MessageListeners. This issue does not affect users of the IBM MQ V8.0.0.5 (and later) classes for JMS, after APAR IT11291. Platforms affected: MultiPlatform **************************************************************** PROBLEM DESCRIPTION: When an MQ classes for JMS application attempts to close either a JMS MessageConsumer or JMS QueueReceiver object that has a JMS MessageListener configured, such that messages are consumed asynchronously, then the close API call will block until any in-progress or pending message delivery to the JMS MessageListener has completed. This also includes any messages that have just been received under an MQ syncpoint but have not yet been delivered to the MessageListener. The MQ classes for JMS use an internal thread, known as the "DispatchThread" to deliver messages to application's JMS MessageListener. As part of stopping a MessageConsumer or QueueReceiver, the MQ classes for JMS notify the DispatchThread of the pending close request to ensure it completes delivery of pending messages consumed under syncpoint. The application thread blocks and waits for the DispatchThread to notify it that the message delivery work has completed before it stops, which allows the close processing to continue. In the case where a request to shutdown (quiesce) the queue manager has been made, the queue manager notifies the MQ classes for JMS that it is ending and is no longer accepting new connections. A "notification" type message is put on an internal queue (known as a ProxyQueue), which is processed by the DispatchThread. When the application thread notifies the DispatchThread of the pending MessageConsumer close call, the DispatchThread checks for messages that have just consumed under an MQ syncpoint that have also been put to this internal queue. It finds that the queue is not empty (as it contains the notification from the queue manager) but the DispatchThread assumes the internal queue contains a valid MQ message with MQMD header. If the JMS Session is not transacted (i.e., false was passed as the first argument in the Connection.createSession(boolean, int) API call), then whether or not the message was consumed under an MQ syncpoint depends on the persistence of the message. In this case, the DispatchThread attempted to get the MQMD of the message in order to check the value of the MQMD's "Persistence" field. However, because the queue contained a notification from the queue manager and not an MQ message, then no MQMD existed. Because of the null MQMD, the operation to query the Persistence threw a java.lang.NullPointerException which caused the DispatchThread to generate an FFDC and end. This meant that it did not notify the waiting application thread, which was closing the MessageConsumer, and so this thread hung indefinitely.
Problem conclusion
The MQ classes for JMS have been updated such that any message (either an actual MQ message or a notification) held on the internal ProxyQueue is delivered to the application's JMS MessageListener before a MessageConsumer.close() call can complete, regardless of whether the message is persistent, non-persistent or consumed under syncpoint. Therefore, the DispatchThread no longer needs to check the Persistence field of the MQMD when making this determination and can continue to process either actual MQ messages or notifications correctly, before notifying the waiting application thread closing the MessageConsumer when message delivery has completed. Please note that the changes in the IBM MQ V8 and V9 product releases made under this APAR are only internal serviceability changes and not functional changes. --------------------------------------------------------------- The fix is targeted for delivery in the following PTFs: Version Maintenance Level v7.5 7.5.0.8 v8.0 8.0.0.7 v9.0 CD 9.0.2 v9.0 LTS 9.0.0.1 The latest available maintenance can be obtained from 'WebSphere MQ Recommended Fixes' http://www-1.ibm.com/support/docview.wss?rs=171&uid=swg27006037 If the maintenance level is not yet available information on its planned availability can be found in 'WebSphere MQ Planned Maintenance Release Dates' http://www-1.ibm.com/support/docview.wss?rs=171&uid=swg27006309 ---------------------------------------------------------------
Temporary fix
Comments
APAR Information
APAR number
IT18135
Reported component name
WMQ BASE MULTIP
Reported component ID
5724H7241
Reported release
750
Status
CLOSED PER
PE
NoPE
HIPER
YesHIPER
Special Attention
NoSpecatt / Xsystem
Submitted date
2016-11-28
Closed date
2016-12-21
Last modified date
2017-06-01
APAR is sysrouted FROM one or more of the following:
APAR is sysrouted TO one or more of the following:
Fix information
Fixed component name
WMQ BASE MULTIP
Fixed component ID
5724H7241
Applicable component levels
R750 PSY
UP
[{"Line of Business":{"code":"LOB45","label":"Automation"},"Business Unit":{"code":"BU053","label":"Cloud & Data Platform"},"Product":{"code":"SSFKSJ","label":"WebSphere MQ"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"7.5"}]
Document Information
Modified date:
26 August 2021