IC81953: HANG ENCOUNTERED IN WEBSPHERE MQ V7.1 RESOURCE ADAPTER WHEN QUEUE MANAGER CHANNEL IS RESTARTED DURING MESSAGE FLOW

Fix Pack 7.1.0.1 for WebSphere MQ V7.1
WebSphere MQ V7.1 for i5/OS Fix Pack 7.1.0.1
Fix Pack 7.1.0.2 for WebSphere MQ V7.1

APAR status

• Closed as program error.

Error description

• A thread hang occurs intermittently in WebSphere Server when
  the WebSphere MQ v7.1 Resource Adapter actively process
  messages for an Activation Specification the queue manager's
  channel is stopped and restarted.
  
  The application server's SystemOut.log reports a thread
  timeout, with the message:
  
  [10/12/11 9:09:56:198 BST] 00000006 TimeoutManage I
  WTRN0124I: When the timeout occurred the thread with which the
  transaction is, or was most recently, associated was
  Thread[WMQJCAResourceAdapter : 1,5,main]. The stack trace of
  this thread when the timeout occurred was:
  
  com.ibm.mq.connector.inbound.MessageEndpointDeployment.onExcept
  ion(MessageEndpointDeployment.java:788)
  com.ibm.mq.connector.inbound.AbstractWorkImpl.run(AbstractWorkI
  mpl.java:275)
  com.ibm.ejs.j2c.work.WorkProxy.run(WorkProxy.java:608)
  com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:1650)
  
  
  An FFDC is also generated, which contains data of the form:
  
  Product          :-
  Date/Time        :- Wed Oct 12 09:09:56 BST
  System time      :-
  Operating System :-
  UserID           :-
  Java Vendor      :- IBM
  Java Version     :- 2.
  
  Source Class     :-
  .        com.ibm.mq.connector.inbound.MessageEndpointDeployment
  Source Method    :- waitForServerSessinExit()
  ProbeID          :- JCA02007
  Thread           :- name=Thread-90 priority=5 group=main
  .              ccl=com.ibm.ws.bootstrap.ExitClassLoader@3f5e72f
  
  .                             Data
  .                             ----
  
  |   sessionsRemaining  :-  1
  |   timeout            :-  120000
  

Local fix

Problem summary

• ****************************************************************
  USERS AFFECTED:
  Users of the WebSphere MQ v7.1 Resource Adapter, who are using
  Activation Specifications.
  
  This includes users of WebSphere Application Server.
  
  Platforms affected:
  All Distributed (iSeries, all Unix and Windows) +Java +Java zOS
  ****************************************************************
  PROBLEM SUMMARY:
  When the queue manager's channel is restarted while an
  Activation Specification was processing messages from a queue
  over that channel, a thread hang may have been experienced.  A
  javacore taken when the system was in this state would show two
  threads with the following stack traces:
  
  "WMQJCAResourceAdapter : 1" J9VMThread:0x000000000196DF00,
  j9thread_t:0x000004000B40B6E0, java/lang
  /Thread:0x000000001254AC68, state:B, prio=5
  (native thread ID:0x3772, native priority:0x5, native
  policy:UNKNOWN)
  (native stack address range from:0x0000040013AC0000,
  to:0x0000040013B00000, size:0x40000)
  Blocked on: com/ibm/ejs/container
  /MessageEndpointFactoryImpl@0x0000000011C36AE8 Owned by:
  "Thread-90" (J9VMThread:0x0000000001B32C00, java/lang
  /Thread:0x000000001DC90010)
  
  Java callstack:
  at com/ibm/mq/connector/inbound/MessageEndpointDeployment.onExc
  eption(MessageEndpointDeployment.java:788)
  at com/ibm/mq/connector/inbound/AbstractWorkImpl.run(AbstractWo
  rkImpl.java:275)
  at com/ibm/ejs/j2c/work/WorkProxy.run(WorkProxy.java:608)
  at com/ibm/ws/util/ThreadPool$Worker.run(ThreadPool.java:1650)
  
  
  
  "Thread-90" J9VMThread:0x0000000001B32C00,
  j9thread_t:0x0000040018FD5E40, java/lang
  /Thread:0x000000001DC90010, state:CW, prio=5
  (native thread ID:0x4BF1, native priority:0x5, native
  policy:UNKNOWN)
  (native stack address range from:0x000004001B630000,
  to:0x000004001B670000, size:0x40000)
  Waiting on: com/ibm/mq/connector/inbound
  /ServerSessionImpl$IsInUseLock@0x000000001DCA5C08 Owned by:
  <unowned>
  
  Java callstack:
  at java/lang/Object.wait(Native Method)
  at java/lang/Object.wait(Object.java:167)
  at com/ibm/mq/connector/inbound/ServerSessionImpl.close(ServerS
  essionImpl.java:285)
  at com/ibm/mq/connector/inbound/ServerSessionPoolImpl.closeInte
  rnal(ServerSessionPoolImpl.java:652)
  at com/ibm/mq/connector/inbound/ServerSessionPoolImpl.close(Ser
  verSessionPoolImpl.java:590)
  at com/ibm/mq/connector/inbound/MessageEndpointDeployment.stopD
  eliveryASF(MessageEndpointDeployment.java:560)
  at com/ibm/mq/connector/inbound/MessageEndpointDeployment.stop(
  MessageEndpointDeployment.java:459)
  at com/ibm/mq/connector/inbound/MessageEndpointDeployment.onExc
  eption(MessageEndpointDeployment.java:803)
  at com/ibm/mq/connector/inbound/MessageEndpointDeployment.onExc
  eption(MessageEndpointDeployment.java:763)
  at com/ibm/msg/client/jms/internal/JmsProviderExceptionListener
  .run(JmsProviderExceptionListener.java:422)
  at com/ibm/msg/client/commonservices/workqueue/WorkQueueItem.ru
  nTask(WorkQueueItem.java:209)
  at com/ibm/msg/client/commonservices/workqueue/SimpleWorkQueueI
  tem.runItem(SimpleWorkQueueItem.java:100)
  at com/ibm/msg/client/commonservices/workqueue/WorkQueueItem.ru
  n(WorkQueueItem.java:224)
  at com/ibm/ws/wmqcsi/workqueue/WorkQueueManagerImpl$WorkQueueRu
  nnable.run(WorkQueueManagerImpl.java:550)
  at java/lang/Thread.run(Thread.java:769)
  

Problem conclusion

• The code has been modified to prevent multiple threads which are
  processing messages for an Activation Specification from
  reaching the deadlocked state, when the message flow between
  the queue manager and resource adapter is interrupted.
  
  ---------------------------------------------------------------
  The fix is targeted for delivery in the following PTFs:
  
                     v7.1
  Platform           Fix Pack 7.1.0.2
  --------           --------------------
  Windows            U200351
  AIX                U853019
  HP-UX (Itanium)    U853052
  Solaris (SPARC)    U853048
  Solaris (x86-64)   U853054
  iSeries            7.1.0.2
  Linux (x86)        U853049
  Linux (x86-64)     U853053
  Linux (zSeries)    U853050
  Linux (Power)      U853051
  zOS                7.1.0.2
  
  The latest available maintenance can be obtained from
  'WebSphere MQ Recommended Fixes'
  http://www-1.ibm.com/support/docview.wss?rs=171&uid=swg27006037
  
  If the maintenance level is not yet available information on
  its planned availability can be found in 'WebSphere MQ
  Planned Maintenance Release Dates'
  http://www-1.ibm.com/support/docview.wss?rs=171&uid=swg27006309
  ---------------------------------------------------------------
  

Temporary fix

Comments

APAR Information

• APAR is sysrouted FROM one or more of the following:

• APAR is sysrouted TO one or more of the following:

Fix information

• Fixed component name

  WMQ WINDOWS V7

• Fixed component ID

  5724H7220

Applicable component levels

• R710 PSY

     UP