IZ83415: WMQ V7: QUEUE MANAGER HANGS, XC307070, XLSREQUESTMUTEX, XECL_W_LONG_LOCK_WAIT

WebSphere MQ V7.0 Fix Pack 7.0.1.4
WebSphere MQ V7.0.1 for i5/OS Fix Pack 7.0.1.4

APAR status

• Closed as program error.

Error description

• A WebSphere MQ V7 queue manager hangs.
  
  FDCs are produced that have the following characteristics:
  
   Probe Id          :- XC307070
   Component         :- xlsRequestMutex
   Major Errorcode   :- xecL_W_LONG_LOCK_WAIT
   Probe Description :- AMQ6150: WebSphere MQ semaphore is busy.
   Comment1          :- OwningProcess(2261) Status(ACTIVE)
  
  One of the FDC files may have the following MQM Function Stack:
  
  zmuThreadMain
  zmuCheckpointTask
  alsCheckPointLoop
  atmReduceLogHead
  attLockOldestTran
  atmLockDataMutex
  xcsRequestMutexSem
  xlsRequestMutex
  xcsFFST
  
  Another FDC may have the following MQM Function Stack:
  
  zlaMainThread
  zlaProcessMessage
  zlaProcessMQIRequest
  zlaMQBACK
  zsqMQBACK
  kpiSyncPoint
  apiSyncPoint
  atmSyncPoint
  atxRollback
  atxPerformRollback
  aqmSyncPointRollback
  aqhSyncPointAction
  aqhAddMsg
  kpiTickle
  xlsRequestMutex
  xcsFFST
  
  Yet another FDC may have this MQM Function Stack:
  
  zlaMainThread
  zlaProcessMessage
  zlaProcessMQIRequest
  zlaMQGETM
  zsqMQGETM
  kpiMQGETM
  apiSyncPointCheck
  atmSyncPointCheck
  atmLockTTMutex
  xcsRequestMutexSem
  xlsRequestMutex
  xcsFFST
  
  MQ processes will be seen to have abnormally high CPU usage with
  amqzmuc0 and amqzlaa0 being particularly affected.
  
  It is also possible to be affected by this deadlock problem
  after hitting the problem fixed by IV10234.  You may see the
  following MQ stacks from two deadlocked threads or processes:
  
  Thread/process 1:
  
  zlaMainThread
  kpiTerminate
  kpiMQDISC
  kqiCloseIt
  kqiCloseQueue
  kqiUnregisterConsumer
  kqiRemoveFromWaiterChain
  xlsRequestMutex
  xcsFFST
  
  Thread/process 2:
  
  zlaMainThread
  zlaProcessMessage
  zlaProcessMQIRequest
  zlaMQPUT
  zsqMQPUT
  kpiMQPUT
  kqiPutIt
  kqiPutMsgSegments
  apiPutMessage
  aqmPutMessage
  aqhPutMessage
  aqhAddMsg
  kpiTickle
  xlsRequestMutex
  xcsFFST
  
  A further thread or process will show
  MQRC_SYNCPOINT_NOT_AVAILABLE from zstMQGETM in the FDC trace
  history.  You will see this you have encountered the problem
  fixed by APAR IV10234.
  

Local fix

Problem summary

• ****************************************************************
  USERS AFFECTED:
  Users that have WebSphere MQ systems which do a lot of
  transactional work may have a higher chance of hitting this
  problem.  It is also possible to hit the problem if the issue
  fixed by APAR IV10234 has recently occurred.
  
  Platforms affected:
  All Distributed (iSeries, all Unix and Windows)
  ****************************************************************
  PROBLEM DESCRIPTION:
  A deadlock between three WebSphere MQ threads caused the queue
  manager to hang.  Three WebSphere MQ locks were taken by the
  three threads in an order which caused a circular dependency
  and deadlock.
  

Problem conclusion

• A piece of code was moved so that the order that the locks were
  taken meant that this deadlock could no longer occur.
  
  ---------------------------------------------------------------
  The fix is targeted for delivery in the following PTFs:
  
  Version    Maintenance Level
  v7.0       7.0.1.4
  
  The latest available maintenance can be obtained from
  'WebSphere MQ Recommended Fixes'
  http://www-1.ibm.com/support/docview.wss?rs=171&uid=swg27006037
  
  If the maintenance level is not yet available information on
  its planned availability can be found in 'WebSphere MQ
  Planned Maintenance Release Dates'
  http://www-1.ibm.com/support/docview.wss?rs=171&uid=swg27006309
  ---------------------------------------------------------------
  

Temporary fix

Comments

APAR Information

• APAR is sysrouted FROM one or more of the following:

• APAR is sysrouted TO one or more of the following:

Fix information

• Fixed component name

  WMQ SOL SPARC

• Fixed component ID

  5724H7223

Applicable component levels

• R701 PSY

     UP