IBM Support

IT26587: IBM MQ Classes for JMS application deadlocks when JMS trace enabled

Subscribe to this APAR

By subscribing, you receive periodic emails alerting you to the status of the APAR, along with a link to the fix after it becomes available. You can track this item individually or track all items by product.

Notify me when this APAR changes.

Notify me when an APAR for this component changes.

 

APAR status

  • Closed as program error.

Error description

  • An IBM MQ V9.1.0.0 classes for JMS application is run with trace
    enabled. After running for a period of time, a deadlock occurs.
    
    A Javacore taken when the deadlock occurs contains the following
    information:
    
    ****************************************************************
    **************
      Deadlock detected!!
    
     Thread "<Thread 1>"
       is waiting for:
         java/util/Collections$SynchronizedMap@<object reference>
       which is owned by:
     Thread "<Thread 2>"
       which is waiting for:
         java/util/logging/FileHandler@<object reference>
       which is owned by:
     Thread "<Thread 1>"
    ****************************************************************
    **************
    "<Thread1>"
        :    :    :    :    :    :    :    :    :    :    :
      Blocked on: java/util/Collections$SynchronizedMap@<object
    reference> Owned by: "<Thread2>"
        :    :    :    :    :    :    :    :    :    :    :
          Java callstack:
              at
    java/util/Collections$SynchronizedMap.entrySet(Collections.java:
    2624(Compiled Code))
              at
    com/ibm/msg/client/jms/admin/JmsDestinationImpl.hashCode(JmsDest
    inationImpl.java:795(Compiled Code))
              at
    com/ibm/msg/client/wmq/common/internal/WMQPropertyContext.hashCo
    de(WMQPropertyContext.java:184(Compiled Code))
              at
    com/ibm/msg/client/wmq/common/internal/WMQDestination.hashCode(W
    MQDestination.java:4842)
              at java/lang/Object.toString(Object.java:161(Compiled
    Code))
              at
    com/ibm/msg/client/commonservices/j2se/trace/TraceUtils$GenericF
    ormatAdapter.formatObject(TraceUtils.java:492(Compiled Code))
              :    :    :    :    :    :    :    :    :    :    :
              at
    java/util/logging/StreamHandler.publish(StreamHandler.java:222(C
    ompiled Code))
                 (entered lock:
    java/util/logging/FileHandler@0x101BEE28, entry count: 2)
              at
    java/util/logging/FileHandler.publish(FileHandler.java:693(Compi
    led Code))
                 (entered lock:
    java/util/logging/FileHandler@0x101BEE28, entry count: 1)
              at
    java/util/logging/Logger.log(Logger.java:749(Compiled Code))
              :    :    :    :    :    :    :    :    :    :    :
              at
    com/ibm/msg/client/commonservices/j2se/trace/DefaultTracer.trace
    Data(DefaultTracer.java:357(Compiled Code))
              :    :    :    :    :    :    :    :    :    :    :
    
    "<Thread2>"
        :    :    :    :    :    :    :    :    :    :    :
    Blocked on: java/util/logging/FileHandler@0x101BEE28 Owned by:
    "Thread-14" (J9VMThread:0x7652DD00, java/lang/Thread:0x105A9600)
        :    :    :    :    :    :    :    :    :    :    :
          Java callstack:
              at
    java/util/logging/FileHandler.publish(FileHandler.java:690(Compi
    led Code))
                 (entered lock:
    java/util/logging/FileHandler@0x101BEE28, entry count: 1)
              at
    java/util/logging/Logger.log(Logger.java:749(Compiled Code))
              :    :    :    :    :    :    :    :    :    :    :
              at
    com/ibm/msg/client/commonservices/trace/Trace.exit(Trace.java:84
    3(Compiled Code))
              at
    com/ibm/msg/client/wmq/common/internal/WMQDestination.mapFromCan
    onical(WMQDestination.java:4478(Compiled Code))
              at
    com/ibm/msg/client/jms/internal/JmsPropertyContextImpl.getObject
    Property(JmsPropertyContextImpl.java:587(Compiled Code))
              at
    com/ibm/msg/client/jms/internal/JmsReadablePropertyContextImpl.g
    etStringProperty(JmsReadablePropertyContextImpl.java:334(Compile
    d Code))
              at
    com/ibm/msg/client/jms/admin/JmsDestinationImpl.hashCode(JmsDest
    inationImpl.java:801(Compiled Code))
                 (entered lock:
    java/util/Collections$SynchronizedMap@0x10FB8F98, entry count:
    1)
              at
    com/ibm/msg/client/wmq/common/internal/WMQPropertyContext.hashCo
    de(WMQPropertyContext.java:184(Compiled Code))
              at
    com/ibm/msg/client/wmq/common/internal/WMQDestination.hashCode(W
    MQDestination.java:4842)
              :    :    :    :    :    :    :    :    :    :    :
    ****************************************************************
    **************
    

Local fix

Problem summary

  • ****************************************************************
    USERS AFFECTED:
    This issue affects users of:
    
    - The IBM MQ V9.0.0.3 (and later) classes for JMS.
    - The IBM MQ V9.0.0.3 (and later) resource adapter.
    
    - The IBM MQ V9.1.0.0 classes for JMS.
    - The IBM MQ V9.1.0.0 resource adapter.
    
    who are asked to collect trace by IBM Support.
    
    
    Platforms affected:
    MultiPlatform
    
    ****************************************************************
    PROBLEM DESCRIPTION:
    APAR IT23101 modified the IBM MQ classes for JMS and resource
    adapter to add some additional locking to the internal method:
    
      JmsDestinationImpl.hashcode()
    
    in order to prevent two threads accessing an internal Map of
    properties on a JMS destination at the same time.  As a result
    of this change, it was possible for a deadlock to occur if IBM
    MQ messaging client trace was enabled, and two or more threads
    were trying to trace information about the same JMS destination
    at the same time. The following sequence of events would lead up
    to the deadlock:
    
    - Thread 1 locked a java/util/logging/FileHandler object, in
    order to write information about the JMS destination to a trace
    file.
    - Thread 2 tried to construct a String representation of the
    same JMS destination, by calling the method
    Destination.toString(). The JMS destination was stored
    internally in a WMQDestination object. As the WMQDestination
    class did not provide a toString() method, the attempt to
    construct a String representing the JMS destination resulted in
    the method:
    
        WMQDestination.hashCode()
    
    being called. This method, in turn, called the internal method:
    
        JmsDestinationImpl.hashCode()
    
    which locked an java/util/Collections$SynchronizedMap object
    (added as part of APAR IT23101) in order to read the properties
    of the JMS destination.
    
    - Next, Thread 1 tried to read the properties of the JMS
    destination. However, it became blocked as it required access to
    the java/util/Collections$SynchronizedMap object that was locked
    by Thread 2.
    
    - While processing the properties of the JMS destination, Thread
    2 tried to trace out the property values. In order to do this,
    it required access to the java/util/logging/FileHandler object
    which was locked by Thread 1.
    

Problem conclusion

  • The IBM MQ classes for JMS have been updated so that the
    internal WMQDestination class now has a toString() method, which
    returns a String containing a URI representing the JMS
    destination.
    
    When information about a JMS destination needs to be written out
    to a trace file, the new WMQDestination.toString() method is
    called, rather than WMQDestination.hashCode().  Because the
    WMQDestination.toString() method does not take any internal
    locks, one thread is able to trace out information about the JMS
    destination at the same time as another thread is reading the
    destination's properties, which prevents the deadlock from
    occurring.
    
    ---------------------------------------------------------------
    The fix is targeted for delivery in the following PTFs:
    
    Version    Maintenance Level
    v9.0 LTS   9.0.0.6
    v9.1 CD    9.1.1
    v9.1 LTS   9.1.0.1
    
    The latest available maintenance can be obtained from
    'WebSphere MQ Recommended Fixes'
    http://www-1.ibm.com/support/docview.wss?rs=171&uid=swg27006037
    
    If the maintenance level is not yet available information on
    its planned availability can be found in 'WebSphere MQ
    Planned Maintenance Release Dates'
    http://www-1.ibm.com/support/docview.wss?rs=171&uid=swg27006309
    ---------------------------------------------------------------
    

Temporary fix

Comments

APAR Information

  • APAR number

    IT26587

  • Reported component name

    IBM MQ BASE M/P

  • Reported component ID

    5724H7261

  • Reported release

    900

  • Status

    CLOSED PER

  • PE

    NoPE

  • HIPER

    NoHIPER

  • Special Attention

    NoSpecatt / Xsystem

  • Submitted date

    2018-10-11

  • Closed date

    2018-10-16

  • Last modified date

    2018-10-16

  • APAR is sysrouted FROM one or more of the following:

  • APAR is sysrouted TO one or more of the following:

Fix information

  • Fixed component name

    IBM MQ BASE M/P

  • Fixed component ID

    5724H7261

Applicable component levels

[{"Business Unit":{"code":"BU053","label":"Cloud & Data Platform"},"Product":{"code":"SSYHRD","label":"IBM MQ"},"Component":"","ARM Category":[],"Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"9.0","Edition":"","Line of Business":{"code":"LOB36","label":"IBM Automation"}}]

Document Information

Modified date:
16 October 2018