A fix is available
APAR status
Closed as program error.
Error description
A javacore dump has a Garbage Collector (GC) thread failing in the MQ finalizer: Current thread ---------------------- "Finalizer thread" J9VMThread:0x00000048869AC200, j9thread_t:0x0000004885FE7820, java/lang/Thread:0x000000488EBCC6B0, state:R, prio=5 java/lang/Thread getId:0x169, isDaemon:true) Heap bytes allocated since last GC cycle=2450088 (0x2562A8) Java callstack: at com/ibm/mq/jmqi/local/internal/base/Native.MQCLOSE (Native Method) at com/ibm/mq/jmqi/local/LocalMQ.MQCLOSE (LocalMQ.java:1978) at com/ibm/mq/MQSESSION.MQCLOSE(MQSESSION.java:801) at com/ibm/mq/MQManagedObject.close(MQManagedObject.java:407) (entered lock: com/ibm/mq/MQQueue@0x0000004891DF1B78, entry count: 2) at com/ibm/mq/MQQueue.close(MQQueue.java:960) (entered lock: com/ibm/mq/MQQueue@0x0000004891DF1B78, entry count: 1) at com/ibm/mq/MQManagedObject.finalize(MQManagedObject.java:444) at java/lang/J9VMInternals.runFinalize(J9VMInternals.java:489 (Compiled Code)) No native callstack available on this platform A JMS trace shows that an MQQueue object had thrown an MQException containing WebSphere MQ Reason Code 2019 ( MQRC_HOBJ_ERROR ). When this MQException was thrown: - The application did not try to close the MQQueue object. - This then caused the WebSphere MQ classes for Java to try to close the MQQueue object during disconnect, which failed with the same MQException. - The Finalizer thread then tried to close the MQQueue object, which caused an ABEND. Additional Symptom(s) Search Keyword(s): . On z/OS, an LE stack in a CICS transaction dump shows the following: 14 __zerro +00001072 15 __zerros +00000350 16 CEEHDSP +00004450 17 CEEOSIGJ +000009B2 18 CELQHROD +00000266 19 CEEOSIGG +00000000 20 CELQHROD +00000266 21 CSQC3CLS +00000150 Exception 22 Java_com_ibm_mq_jmqi_local_internal_base_Native_MQCLOSE +000000F2 23 RUNCALLINMETHOD +00000000 24 callStaticVoidMethodV +00000040 . The abend occurs when CSQCSTUB calls the DFHRMCAL macro to invoke the MQ true as part of an MQCLOSE call issued on the finalizer thread. . 5655R3600 R100 ABEND0C1 ABEND0C4 DFHSJ0001 AN ABEND (CODE ---/AKEB) HAS OCCURRED AT OFFSET X'192E' IN MODULE DFHSJIN.
Local fix
Update your application to close the queue even if the close fails. This will remove it from the internal list of open objects. For example, if your logic is try { Create SendSession object. Create ReplaySession object. Do some work. Close the SendSession object. Close the ReplaySession object. Page 80 of 129 } catch (Exception e) { Log the exception. } change it to Define SendSession object. Define ReplaySession object. try { Create the SendSession object. Create the ReplaySession object. Do some work. } catch (Exception e) { Log the exception. } finally { Close the SendSession object. Close the ReplaySession object. }
Problem summary
**************************************************************** USERS AFFECTED: This issue affects users of: - The WebSphere MQ V7.1 classes for Java. - The WebSphere MQ V8 classes for Java. who have applications running in a CICS Transaction Server for z/OS 5.1.0 JVM server environment that connect to a WebSphere MQ queue manager using the BINDINGS transport. Platforms affected: MultiPlatform **************************************************************** PROBLEM DESCRIPTION: When using the WebSphere MQ classes for Java, every MQQueueManager object created by an application maintains internal lists of all of the distribution lists, processes, queues and topics that have been opened using it. When an application calls one of the methods shown below: - MQQueueManager.accessQueue(String, int) - MQQueueManager.accessQueue(String, int, String, String, String) and successfully opens a queue on the queue manager, an MQQueue object is created, marked as being in an "open" state and returned to the application. The MQQueue object is also added to the internal list of queues maintained by the MQQueueManager. When the method: - MQQueue.close() is called, the WebSphere MQ classes for Java will flow an MQCLOSE to the queue manager. If the MQCLOSE call completes successfully, the state of the MQQueue object changes to "closed" and it is removed from the internal list of queues. Similar processing takes place when an application calls the following methods: - MQQueueManager.accessDistributionList(MQDistributionListItem[], int) - MQQueueManager.accessDistributionList(MQDistributionListItem[], int, String) - MQDistributionList.close() - MQQueueManager.accessProcess(String, int) - MQQueueManager.accessProcess(String, int, String, String) - MQProcess.close() - MQQueueManager.accessTopic(MQDestination,String, String, int) - MQQueueManager.accessTopic(MQDestination,String, String, int, String) - MQQueueManager.accessTopic(MQDestination,String, String, int, String, String) - MQQueueManager.accessTopic(MQDestination,String, String, int, String, String, Hashtable) - MQQueueManager.accessTopic(String,String, int, int) - MQQueueManager.accessTopic(String,String, int, int, String) - MQQueueManager.accessTopic(String,String, int, int, String, String) - MQQueueManager.accessTopic(String,String, int, int, String, String, Hashtable) - MQTopic.close() When an application calls: - MQQueueManager.disconnect() the WebSphere MQ classes for Java will look at the internal lists of distribution lists, processes, queues and topics associated with that MQQueueManager object, and close each one in turn. After the objects have been closed, they are removed from the internal lists. Due to the logic inside the MQQueueManager.disconnect() method, if an attempt to close an MQQueue object in the internal list of queues fails for some reason, then no more MQQueue objects will be closed. However, the internal list of queues would be cleared. This meant that any MQQueue objects that had not been closed by the MQQueueManager.disconnect() method by the time the list was cleared are be eligible for garbage collection by the Java Runtime Environment that the WebSphere MQ classes for Java were running in. The garbage collection processing runs on a Finalizer thread provided by the Java Runtime Environment, and would ensure that an MQCLOSE API call is sent to the queue manager The same logic also affects the way the MQQueueManager.disconnect() method closes MQDistributionList objects stored in the internal list of MQDistributionList, and and MQProcess objects stored in the internal list of processes. For MQTopic objects, the logic in the MQQueueManager.disconnect() method ensures that an attempt is made to close all of the MQTopic objects stored in the internal list of topics, even if an attempt to close one or more of the MQTopic objects failed with an error. When running inside a JVM server environment provided by CICS Transaction Server for z/OS 5.1.0, only CICS managed threads are allowed to make WebSphere MQ API calls. The Finalizer thread provided by the Java Runtime Environment is not considered a CICS managed thread. If an attempt to close an MQDistributionList, MQProcess or MQQueue object was peformed by the Finalizer thread, then an ABEND would occur when the thread flowed the MQCLOSE API call to the queue manager, as the thread was not permitted to make the call.
Problem conclusion
Two changes have been made to the WebSphere MQ classes for Java as part of this APAR: 1) The MQQueueManager.disconnect() method will now attempt to close all of the MQDistributionList, MQProcess and MQQueue objects stored inside the internal lists, even if one or more of these attempts fails with an error. 2) If an attempt to close an MQDistributionList, MQProcess, MQQueueManager or MQQueue method by an application fails for some reason, and the object is eligible for garbage collection, the Finalizer thread will no longer flow an MQCLOSE API call to the queue manager. If an attempt to close one of these objects has failed once, then all subsequent attempts to close it by issuing an MQCLOSE API call will also fail, which means that the second attempt is redundant. Not attempting a second MQCLOSE attempt is a safe operation, as the queue manager will clean up the open handles for the distribution list, process, queue manager and queue objects once the application finishes. Both of these changes affect the WebSphere MQ V7.1, V7.5 and V8 classes for Java on all platforms. --------------------------------------------------------------- The fix is targeted for delivery in the following PTFs: Version Maintenance Level v7.1 7.1.0.7 v7.5 7.5.0.5 v8.0 8.0.0.2 The latest available maintenance can be obtained from 'WebSphere MQ Recommended Fixes' http://www-1.ibm.com/support/docview.wss?rs=171&uid=swg27006037 If the maintenance level is not yet available information on its planned availability can be found in 'WebSphere MQ Planned Maintenance Release Dates' http://www-1.ibm.com/support/docview.wss?rs=171&uid=swg27006309 ---------------------------------------------------------------
Temporary fix
Comments
APAR Information
APAR number
IV65687
Reported component name
WMQ AIX V7
Reported component ID
5724H7221
Reported release
710
Status
CLOSED PER
PE
NoPE
HIPER
NoHIPER
Special Attention
NoSpecatt
Submitted date
2014-10-08
Closed date
2014-11-28
Last modified date
2014-11-28
APAR is sysrouted FROM one or more of the following:
APAR is sysrouted TO one or more of the following:
Fix information
Fixed component name
WMQ AIX V7
Fixed component ID
5724H7221
Applicable component levels
R710 PSY
UP
[{"Line of Business":{"code":"LOB45","label":"Automation"},"Business Unit":{"code":"BU053","label":"Cloud & Data Platform"},"Product":{"code":"SSFKSJ","label":"WebSphere MQ"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"7.1"}]
Document Information
Modified date:
08 March 2021