IC76463: WMQ V7: A NON-STARTED MDB ATTEMPTS TO PROCESS MESSAGES, RESULTING ON JBOSS IN THE MESSAGE:"THE CONTAINER IS NOT STARTED"

WebSphere MQ v7.0.1.7 Hypervisor for AIX
WebSphere MQ V7.0 Fix Pack 7.0.1.7
WebSphere MQ V7.0.1 for i5/OS Fix Pack 7.0.1.7

APAR status

• Closed as program error.

Error description

• Message Driven Beans (MDBs) unexpectedly generate FDCs
  that eventually fill the file system when using the WebSphere
  MQ v7.0.1.x Resource Adapter (RA) with JBoss application
  server.
  
  The JBoss error log captures the following:
  
  "JMSCC0034:  The asynchronous delivery mechanism is in an
  inconsistent state in method:  'deliver'.  An unexpected
  sequence of events has been detected during delivery of
  messages to a message-driven bean (MDB).  This may be caused by
  client programming application errors.  Check the application
  server log files for other problems associated with the
  message-driven bean."
  
  The FFDC generated for the MDB is as follows:
  --------------------------------------------
  Product :- IBM WebSphere MQ classes for JMS
  
  Date/Time :-
  System time :-
  Operating System :- Linux
  UserID :- *******
  Java Vendor :- Sun Microsystems Inc.
  Java Version :- 1.5.0_19-b02
  Source Class :-
  com.ibm.msg.client.jms.internal.
  JmsConnectionConsumerImpl$JmsMessageReferenceHandlerImpl
  Source Method :- endDeliverInternal()
  ProbeID :- XJ009003
  Thread :- name=DispatchThread: 720 priority=5
  group=JMSCCThreadPool
  ccl=org.jboss.mx.loading.UnifiedClassLoader3@54640b25{
  url=file:<JBOSS_DEPLOYMENT_DIR>/wmq.jmsra.rar,
  addedOrder=72
  }
  
  Data
  ----
  | Exception :- ExceptionDepth is 2
  | Exception :-
  | | Cause:1 :-
  com.ibm.msg.client.jms.DetailedIllegalStateException:
  JMSCC0034:
  The asynchronous delivery mechanism is in an inconsistent state
  in method:  'deliver'.  An unexpected sequence of events has
  been detected during delivery of messages to a message-driven
  bean (MDB).  This may be caused by client programming
  application errors.  Check the application server log files for
  other problems associated with the message-driven bean.
  ------------------------------------------------------
  
  The MDB later experiences problems connecting to the QMgr and
  trace reveals the below:
  
  ----------------------------
  The container is not started
  [javax.resource.spi.UnavailableException] at:
  
  org.jboss.ejb.plugins.inflow.JBossMessageEndpointFactory.
  createEndpoint(JBossMessageEndpointFactory.java:166)
  com.ibm.mq.connector.inbound.WorkImpl.run(WorkImpl.java:192)
  org.jboss.resource.work.WorkWrapper.execute(WorkWrapper.java:21
  3)
  org.jboss.util.threadpool.BasicTaskWrapper.
  run(BasicTaskWrapper.java:275)
  
  java.lang.Thread.run(Thread.java:619)
  Object ClassLoader =
  org.jboss.mx.loading.UnifiedClassLoader3@77ff92f5{ url=null
  ,addedOrder=2}
  CurrentThread ClassLoader =
  org.jboss.mx.loading.UnifiedClassLoader3@148c0c7a{
  url=file:<JBOSS_DEPLOYMENT_DIR>/wmq.jmsra.rar,
  addedOrder=46
  }
  

Local fix

• Configure the WMQ-RA property 'connectionConcurrency' to have
  the value 1. More details are available at the following
  Information Center document:
  
  WebSphere MQ classes for JMS -> Installation and configuration
  -> The WebSphere MQ resource adapter -> Configuration ->
  Configuration of the ResourceAdapter object
  
  URL:
  
  http://publib.boulder.ibm.com/infocenter/wmqv7/v7r0/topic/com.ib
  m.mq.csqzaw.doc/jm40190_.htm
  

Problem summary

• ****************************************************************
  USERS AFFECTED:
  This issue affects users of application servers which utilise
  the v7 WebSphere MQ JCA Resource Adapter, including WebSphere
  Application Server v7, and employ activation specifications.
  
  Platforms affected:
  All Distributed (iSeries, all Unix and Windows) +Java
  ****************************************************************
  PROBLEM SUMMARY:
  When an activation specification is defined within the JEE
  environment, which uses the v7 WebSphere MQ Resource Adapter
  to trigger the Message-Driven Bean (MDB) from a WebSphere MQ
  destination, the activation specification is started at the
  request of the application server using the JCA API.
  
  If there is a problem during this start up, for example if the
  WebSphere MQ queue manager is not running, then the endpoint
  throws an exception back to the application server to indicate
  that the activation specification did not start.
  
  If the resource adapter is making use of the
  ConnectionConcurrency property (which is the default
  configuration), multiple activation specifications may use the
  same JMS Connection object to connect to the queue manager.
  
  If an activation specification is running, and sharing a JMS
  Connection object with another activation specification which
  fails to start, a code defect meant that should the running
  activation specification encounter a connection problem, the
  subsequent connection reconnection logic would attempt to start
  the endpoint which failed to start, when connectivity to the
  queue manager was resumed.
  
  This left the system in a state where the application server has
  the endpoint in a stopped state, but it had actually been
  started by the reconnection code, and was monitoring the queue
  for messages.
  
  On JBoss where the problem was observed, when a suitable
  message was found on the queue, the message consuming logic
  requested the creation of a ServerSession to process the
  message within the MDB.  When this ServerSession was started,
  JBoss refused to create an endpoint from the EndpointFactory,
  issuing the exception message:
  
  'The container is not started'
  
  This resulted in the message not being consumed, which led to
  the message being repeatedly observed as the activation
  specification attempted to process it again and again.
  
  
  A workaround for the issue is to set the WebSphere MQ Resource
  Adapter property ConnectionConcurrency to have the value 1.
  This prevents non-started endpoints from being started in
  error.
  

Problem conclusion

• The WebSphere MQ Resource Adapter was changed such that if the
  endpoint does not start, it is removed from the list of
  activation specifications which is sharing the JMS Connection
  object.
  
  This results in the WebSphere MQ Resource Adapter reconnection
  logic not starting an endpoint which failed to start following
  the application server's activation request.
  
  ---------------------------------------------------------------
  The fix is targeted for delivery in the following PTFs:
  
                     v7.0
  Platform           Fix Pack 7.0.1.7
  --------           --------------------
  Windows            U200333
  AIX                U843721
  HP-UX (PA-RISC)    U844089
  HP-UX (Itanium)    U844094
  Solaris (SPARC)    U844090
  Solaris (x86-64)   U844096
  iSeries            tbc_p700_0_1_7
  Linux (x86)        U844091
  Linux (x86-64)     U844095
  Linux (zSeries)    U844092
  Linux (Power)      U844093
  
  The latest available maintenance can be obtained from
  'WebSphere MQ Recommended Fixes'
  http://www-1.ibm.com/support/docview.wss?rs=171&uid=swg27006037
  
  If the maintenance level is not yet available, information on
  its planned availability can be found in 'WebSphere MQ
  Planned Maintenance Release Dates'
  http://www-1.ibm.com/support/docview.wss?rs=171&uid=swg27006309
  ---------------------------------------------------------------
  

Temporary fix

Comments

APAR Information

• APAR is sysrouted FROM one or more of the following:

• APAR is sysrouted TO one or more of the following:

Fix information

• Fixed component name

  WMQ WINDOWS V7

• Fixed component ID

  5724H7220

Applicable component levels

• R700 PSY

     UP