IBM Support

IT09442: IBM MQ V8 JMS APP THAT CREATES/CLOSES MANY CONNECTIONS TO A QMGR CAUSES HIGH CPU, MEMORY UTILIZATION.

Subscribe

You can track all active APARs for this component.

 

APAR status

  • Closed as program error.

Error description

  • When IBM MQ v8 JMS application creates/closes many
    connections to a queue manager, there is gradual CPU usage
    increase over time, and an increase in the JVM memory usage.
    This causes serious impact on the JVM and GC.
    
    Over 30 day period the system/CPU load (as reported by Linux)
    is increasing, from around 0.1-0.5 to 3-5%.
    
    Memory allocation increase, Every time full GC is performed,
    memory goes down, but memory allocation increases, and as a
    consequence, frequency of (full) GCs is increasing.
    
    Once the system is under increased load (even though it's
    idle really) performing a particular user action/request can
    push the system over the line and it then does not recover.
    

Local fix

Problem summary

  • ****************************************************************
    USERS AFFECTED:
    This issue affects users of:
    
    - The IBM MQ V8 allclient JAR file.
    - The IBM MQ V8 classes for JMS.
    - The IBM MQ V8 classes for JMS OSGi bundle.
    - The IBM MQ V8 resource adapter.
    
    who have JMS applications that:
    
    - Connect to a queue manager using either IBM MQ messaging
    provider normal mode, or IBM MQ messaging provider normal mode
    with restrictions, using the CLIENT transport.
    - And connect using an IBM MQ channel instance that has the
    SHARECNV attribute set to a value greater than 1.
    - And create lots of short-lived JMS connections, JMS sessions
    or JMSContexts.
    
    
    Platforms affected:
    MultiPlatform
    
    ****************************************************************
    PROBLEM DESCRIPTION:
    When an IBM MQ classes for JMS application connects to a queue
    manager in either IBM MQ messaging provider normal mode, or IBM
    MQ messaging provider normal mode with restrictions, using the
    CLIENT transport, it can make use of the "conversation sharing"
    functionality provided by the queue manager. This allows
    multiple conversations between the application and the queue
    manager to use the same underlying channel instance (or TCP/IP
    connection, if you prefer to think of it in those terms). The
    maximum number of conversations that can take place over a
    single channel instance is specified by the IBM MQ channel
    property SHARECNV. The default value is 10.
    
    For JMS applications, every JMS Connection starts a conversation
    with the queue manager. Similarly, every JMS Session created by
    an application also starts a conversation with the queue
    manager. Each JMSContext starts two conversations with a queue
    manager.
    
    Every conversation that is started over a channel instance has a
    unique number (or identifier), which increments by 2 - the first
    conversation over a specified channel instance is given the
    number 1, then second conversation over the same channel
    instance has the number 3, and so on. These conversation
    identifiers are generated by the queue manager. If the WebSphere
    MQ classes for JMS are required to start a new conversation on a
    new channel instance, then that conversation will have the
    identifier 1.
    
    The IBM MQ classes for JMS maintain internal lists containing
    information about the conversations that are currently active
    over a channel instance. When a new conversation is started,
    details of that conversation are added to the internal list for
    the appropriate channel instance. When a conversation ends, the
    entry for that conversation is removed from the relevant list.
    
    These lists were defined as arrays, which was keyed off of the
    conversation identifier. When a new conversation was created on
    the channel instance, some information about that conversation
    was put into the array at a position specified by the identifier
    for that conversation.
    
    If an application created lots of short lived conversations over
    a single channel instance, then the following behaviour
    occurred:
    
    - A JMS Connection was created. This resulted in a new channel
    instance being created, as well as a new conversation being
    started. This conversation had identifier 1, and so went into
    position 1 in the array for the channel instance.
    - A short-lived JMS Session was created. A new conversation was
    started with the queue manager, which had conversation
    identifier 3. The array was resized to contain 3 elements, and
    details about this conversation went into position 3 in the
    array.
    - When the JMS Session was closed, the conversation was ended
    and details about it were removed from position 3 in the array.
    - A second short-lived JMS Session was created and a new
    conversation was started with the queue manager. This had
    conversation identifier 5. The array was resized to contain 5
    elements, and the details about the new conversation went into
    position 5 in the array.
    - When this JMS Session was closed, the conversation was ended
    and details about it were removed from position 5 in the array.
    
    If we move forward in time a bit now, the following behaviour
    was seen:
    
    - A short-lived JMS Session was created from the original JMS
    connection (which had conversation identifier 1). A new
    conversation was started with the queue manager, which had
    conversation identifier 505. The array was resized to contain
    505 elements, and information about this conversation went into
    position 505 in the array.
    - When the JMS Session was closed, the conversation was ended
    and details about it were removed from position 505 in the
    array.
    
    And moving forward a bit further in time:
    
    - A short-lived JMS Session was created from the original JMS
    connection (which had conversation identifier 1) and a new
    conversation was started with the queue manager. This had
    conversation identifier 5005. The array was resized to contain
    5005 elements, and details about this conversation went into
    position 5005 in the array.
    - When this JMS Session was closed, the conversation was ended
    and details about it were removed from position 5005 in the
    array.
    
    And again:
    
    - A short-lived JMS Session was created from the original JMS
    connection (which had conversation identifier 1). A new
    conversation was started with the queue manager, which had
    conversation identifier 50005. The array was resized to contain
    50005 elements, and information about the new conversation went
    added to the array in position 50005.
    - When the JMS Session was closed, the conversation was ended
    and details about it were removed from position 50005 in the
    array.
    
    This resulted in the Java Runtime Environment creating a bigger
    array each time a new conversation was started, even though only
    a few elements in the array were actually being used......
    Because of the size of the array, the memory usage of the Java
    Runtime Environment that the application was running in kept
    growing and shrinking, which resulted in high CPU usage as the
    Java Runtime Environment had to perform lots of garbage
    collection to free up enough memory to hold the array.
    

Problem conclusion

  • The IBM MQ classes for JMS have been updated to use sets to
    store information about the conversations associated with the
    object, rather than an array. When a new conversation is
    started, an entry is added to the set. The entry will be removed
    (and the size of the set decremented) when the conversation
    finishes. This ensures that the size of the set will never be
    larger than the maximum number of conversations that can take
    place over the channel instance (as defined by the IBM MQ
    channel attribute SHARECNV).
    
    ---------------------------------------------------------------
    The fix is targeted for delivery in the following PTFs:
    
    Version    Maintenance Level
    v8.0       8.0.0.4
    
    The latest available maintenance can be obtained from
    'WebSphere MQ Recommended Fixes'
    http://www-1.ibm.com/support/docview.wss?rs=171&uid=swg27006037
    
    If the maintenance level is not yet available information on
    its planned availability can be found in 'WebSphere MQ
    Planned Maintenance Release Dates'
    http://www-1.ibm.com/support/docview.wss?rs=171&uid=swg27006309
    ---------------------------------------------------------------
    

Temporary fix

Comments

APAR Information

  • APAR number

    IT09442

  • Reported component name

    WMQ BASE MULTIP

  • Reported component ID

    5724H7251

  • Reported release

    800

  • Status

    CLOSED PER

  • PE

    NoPE

  • HIPER

    NoHIPER

  • Special Attention

    NoSpecatt

  • Submitted date

    2015-06-12

  • Closed date

    2015-09-14

  • Last modified date

    2015-09-14

  • APAR is sysrouted FROM one or more of the following:

  • APAR is sysrouted TO one or more of the following:

Fix information

  • Fixed component name

    WMQ BASE MULTIP

  • Fixed component ID

    5724H7251

Applicable component levels

  • R800 PSY

       UP

[{"Business Unit":{"code":"BU053","label":"Cloud & Data Platform"},"Product":{"code":"SSYHRD","label":"IBM MQ"},"Component":"","ARM Category":[],"Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"8.0.0.0","Edition":"","Line of Business":{"code":"LOB36","label":"IBM Automation"}}]

Document Information

Modified date:
14 September 2015