IBM Support

IC74733: WEBSPHERE MQ MESSAGES LOST DURING APPLICATION SERVER MDB PROCESSING

Subscribe

You can track all active APARs for this component.

 

APAR status

  • Closed as program error.

Error description

  • A Message Driven Bean (MDB) running inside a J2EE application
    server is driven by an activation specification that is using
    the v7 WebSphere MQ Resource Adapter to trigger the MDB with
    messages on a WebSphere MQ queue.
    
    Whilst messages are flowing, the channel being used by the
    activation specification is stopped and restarted.
    Occasionally this results in one of the in-flight messages being
    removed from the WebSphere MQ queue, but not processed by the
    MDB.  The following exception message is seen in the application
    server system output logs:
    
    WorkImpl) [:/44444444] Caught exception:
    java.lang.RuntimeException: JMSCC0110: An exception '
    Message : com.ibm.msg.client.jms.DetailedIllegalStateException:
    JMSWMQ0031: 'MQOPEN' failed for the queue manager when handling
    a poison message.  When requeuing a poison message, 'MQOPEN'
    failed when attempting to inquire about the queue manager.
    Please see the linked exception for more information on the
    failures and/or warnings.
    
    Class :
    class com.ibm.msg.client.jms.DetailedIllegalStateException
    Stack :
    com.ibm.msg.client.wmq.common.internal.Reason
    .reasonToException(Reason.java:496)
    : com.ibm.msg.client.wmq.common.internal.Reason
    .createException(Reason.java:236)
    : com.ibm.msg.client.wmq.internal.WMQPoison
    .checkJmqiCallSuccess(WMQPoison.java:1111)
    : com.ibm.msg.client.wmq.internal.WMQPoison
    .access$2200(WMQPoison.java:84)
    : com.ibm.msg.client.wmq.internal.WMQPoison$QmAttrs
    .<init>(WMQPoison.java:1381)
    : com.ibm.msg.client.wmq.internal.WMQPoison$QmAttrs
    .<init>(WMQPoison.java:1365)
    : com.ibm.msg.client.wmq.internal.WMQPoison
    .<init>(WMQPoison.java:153)
    : com.ibm.msg.client.wmq.internal.WMQConsumerShadow
    .initialisePoison(WMQConsumerShadow.java:1176)
    : com.ibm.msg.client.wmq.internal.WMQConsumerShadow
    .getMsg(WMQConsumerShadow.java:1301)
    : com.ibm.msg.client.wmq.internal.WMQConsumerShadow
    .getMsg(WMQConsumerShadow.java:1211)
    : com.ibm.msg.client.wmq.internal.WMQSyncConsumerShadow
    .receive(WMQSyncConsumerShadow.java:366)
    : com.ibm.msg.client.wmq.internal.WMQSession
    .loadMessageReference(WMQSession.java:1318)
    : com.ibm.msg.client.jms.internal.JmsSessionImpl
    .consume(JmsSessionImpl.java:2940)
    : com.ibm.msg.client.jms.internal.JmsSessionImpl
    .run(JmsSessionImpl.java:2631)
    : com.ibm.mq.jms.MQSession.run(MQSession.java:862)
    : com.ibm.mq.connector.inbound.WorkImpl.run(WorkImpl.java:265)
    : com.ibm.ejs.j2c.work.WorkProxy.run(WorkProxy.java:399)
    : com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:1550)
    
    Caused by [1] --> Message : com.ibm.mq.MQException:
    JMSCMQ0001: WebSphere MQ call failed with compcode '2'
    ('MQCC_FAILED') reason '2202' ('MQRC_CONNECTION_QUIESCING').
    Class : class com.ibm.mq.MQException
    Stack : com.ibm.msg.client.wmq.common.internal.Reason
    .createException(Reason.java:223)
    : com.ibm.msg.client.wmq.internal.WMQPoison
    .checkJmqiCallSuccess(WMQPoison.java:1111)
    : com.ibm.msg.client.wmq.internal.WMQPoison
    .access$2200(WMQPoison.java:84)
    : com.ibm.msg.client.wmq.internal.WMQPoison$QmAttrs
    .<init>(WMQPoison.java:1381)
    : com.ibm.msg.client.wmq.internal.WMQPoison$QmAttrs
    .<init>(WMQPoison.java:1365)
    : com.ibm.msg.client.wmq.internal.WMQPoison
    .<init>(WMQPoison.java:153)
    : com.ibm.msg.client.wmq.internal.WMQConsumerShadow
    .initialisePoison(WMQConsumerShadow.java:1176)
    : com.ibm.msg.client.wmq.internal.WMQConsumerShadow
    .getMsg(WMQConsumerShadow.java:1301)
    : com.ibm.msg.client.wmq.internal.WMQConsumerShadow
    .getMsg(WMQConsumerShadow.java:1211)
    : com.ibm.msg.client.wmq.internal.WMQSyncConsumerShadow
    .receive(WMQSyncConsumerShadow.java:366)
    : com.ibm.msg.client.wmq.internal.WMQSession
    .loadMessageReference(WMQSession.java:1318)
    : com.ibm.msg.client.jms.internal.JmsSessionImpl
    .consume(JmsSessionImpl.java:2940)
    : com.ibm.msg.client.jms.internal.JmsSessionImpl
    .run(JmsSessionImpl.java:2631)
    : com.ibm.mq.jms.MQSession.run(MQSession.java:862)
    : com.ibm.mq.connector.inbound.WorkImpl.run(WorkImpl.java:265)
    : com.ibm.ejs.j2c.work.WorkProxy.run(WorkProxy.java:399)
    : com.ibm.ws.util.ThreaPool$Worker.run(ThreadPool.java:1550)
    

Local fix

Problem summary

  • ****************************************************************
    USERS AFFECTED:
    This problem affects users of the v7 WebSphere MQ Resource
    Adapter who are using activation specifications to drive an MDB
    with messages from a WebSphere MQ queue.
    
    This includes users of the v7 WebSphere Application Server.
    
    Platforms affected:
    All Distributed (iSeries, all Unix and Windows) +Java
    ****************************************************************
    PROBLEM SUMMARY:
    This problem concerned a small timing window between when a
    message had been destructively removed from the source
    WebSphere MQ queue, and before the message was handed off to the
    user MDB's application for processing, that is, before the
    onMessage(Message)method was invoked.
    
    If a problem occurred during the retrieving and subsequent
    processing of the WebSphere MQ message BEFORE calling the
    application's onMessage(Message) method, an exception describing
    the problem would be seen in the application server's log
    files.
    
    It would then be expected that in the transactional environment
    the transaction would subsequently roll-back, making the
    message available again on the queue.  However a code defect
    meant that if the application server's transaction manager was
    notified to commit the destructive get, resulting in the
    message being lost from the system.
    
    This problem could be encountered whenever the v7 WMQ Resource
    Adapter was being used to drive an MDB, and a problem was
    encountered after a destructive get had been completed to
    remove the message from the source queue, but before the
    application's onMessage() method was invoked.  Any exceptions
    that occurred once onMessage() had been invoked would result
    in the application server correctly managing the transactional
    context; for example, when appropriate, the transaction would
    be marked as roll-back and the message made visible again on
    the source queue.
    
    The problem (and the specific stack trace listed above) was
    found by the WebSphere MQ test team, during a test case which
    involved the restarting of the queue manager channel during
    the operation of WebSphere Application Server.  However, there
    are numerous reasons that could result in this message loss
    being encountered.  The following scenarios have resulted in
    this issue being observed:
    
    1) Channel stop initiated after a message with a backout count
    greater than 1 was received. Further communication is required
    with the queue manager to determine the destination backout
    threshold for poison message processing. This communication
    fails due to the channel quiescing, leading to the exception
    stack being printed to the application server log files, and the
    transaction being committed, losing the message.
    
    (2) The Resource Adapter receives a message containing String
    data encoded in a character set that the application server JVM
    cannot process. APAR IC72897 was introduced to change the
    default behaviour of the WMQ JMS client such that MQGET calls
    are performed without using the MQGMO_CONVERT option. This
    results in the messages arriving from the queue manager in
    their native CCSID encoding as stored on the queue. If this
    CCSID is not valid, or the application server JVM cannot
    correlate this to a valid Java CharacterSet, an exception is
    thrown to indicate the conversion failure. The destructive get
    of the message from the source queue is then committed, losing
    the message.
    

Problem conclusion

  • The code was altered such that when an exception occurs between
    getting the message from the queue and invoking the
    onMessage(Message) method, the transaction is marked for
    roll-back.
    
    This subsequent transactional roll-back ensures that the message
    is not lost from the system.
    
    ---------------------------------------------------------------
    The fix is targeted for delivery in the following PTFs:
    
                       v7.0
    Platform           Fix Pack 7.0.1.5
    --------           --------------------
    Windows            U200324
    AIX                U839183
    HP-UX (PA-RISC)    U839621
    HP-UX (Itanium)    U839626
    Solaris (SPARC)    U839622
    Solaris (x86-64)   U839628
    iSeries            tbc_p700_0_1_5
    Linux (x86)        U839623
    Linux (x86-64)     U839627
    Linux (zSeries)    U839624
    Linux (Power)      U839625
    
    The latest available maintenance can be obtained from
    'WebSphere MQ Recommended Fixes'
    http://www-1.ibm.com/support/docview.wss?rs=171&uid=swg27006037
    
    If the maintenance level is not yet available, information on
    its planned availability can be found in 'WebSphere MQ
    Planned Maintenance Release Dates'
    http://www-1.ibm.com/support/docview.wss?rs=171&uid=swg27006309
    ---------------------------------------------------------------
    

Temporary fix

Comments

APAR Information

  • APAR number

    IC74733

  • Reported component name

    WMQ WINDOWS V7

  • Reported component ID

    5724H7220

  • Reported release

    700

  • Status

    CLOSED PER

  • PE

    NoPE

  • HIPER

    NoHIPER

  • Special Attention

    NoSpecatt

  • Submitted date

    2011-02-28

  • Closed date

    2011-03-25

  • Last modified date

    2011-03-25

  • APAR is sysrouted FROM one or more of the following:

  • APAR is sysrouted TO one or more of the following:

Fix information

  • Fixed component name

    WMQ WINDOWS V7

  • Fixed component ID

    5724H7220

Applicable component levels

  • R700 PSY

       UP

[{"Business Unit":{"code":"BU048","label":"IBM Software"},"Product":{"code":"SSCPQ63","label":"APAR \/ Maintenance"},"Component":"","ARM Category":[],"Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"7.0","Edition":"","Line of Business":{"code":"","label":""}}]

Document Information

Modified date:
25 March 2011