IT15584: When MQ generates AMQ9469 cluster messages FDC files are created that dump the cluster cache leading

APAR status

• Closed as program error.

Error description

• IBM MQ V8 generates large FDCs with Probe Id RM191020 for the
  repository manager process (amqrrmfa)
  The FDCs are generated along with the reported AMQ9469 error
  message in the queue manager logs.
  They contain a full dump of the entire cached queue manager
  cluster configuration and can be numerous.
  As a result they can consume large  amounts of disk space.
  Examples:
  
  AMQ9469: Update not received for CLUSRCVR channel TO.QM1
  hosted on queue manager QM1_2016-01-01_11.11.11 in cluster
  CLUSTER1.
  
  EXPLANATION:
  The repository manager detected that the CLUSRCVR channel
  has not been re-published by its owning queue manager. This
  re-publish action should have happened automatically 4 days
  ago, or in the time between then and now.
  
  See the knowledge center for the full error message:
  
  http://www.ibm.com/support/knowledgecenter/en/SSFKSJ_8.0.0/com.i
  bm.mq.ref.doc/amq9.htm
  
      The FDC will show the following:
  
  Probe Id          :- RM191020
  Component         :- rrmMaintainClqMgr
  Program Name      :- amqrrmfa
  Thread            :- 1    Repository
  Probe Type        :- INCORROUT
  Probe Description :- AMQ6125: An internal WebSphere MQ error
  has occurred.
  Comment1          :- Repeated occurrences of AMQ9469
  
  MQM Function Stack
  amqrrfma_main
  rrmMain
  rrmRepository
  rrmGetMsg
  rrmRunTimers
  rrmMaintenance
  rfxEnumCLQMGR
  rrmMaintainClqMgr
  xcsFFST
  

Local fix

• The following environment variables can be exported to reduce
  the size of the FDCs being generated. The queue manager will
  need to be restarted after they are in place.
  .
  Unix:
    export AMQ_RRMFA_MSG_HISTORY=0
    export AMQ_RRMFA_SUPPRESS_CACHEDUMP=1
  Windows:
    set AMQ_RRMFA_MSG_HISTORY=0
    set AMQ_RRMFA_SUPPRESS_CACHEDUMP=1
  .
  

Problem summary

• ****************************************************************
  USERS AFFECTED:
  Users of Queue Manager Cluster configurations with have member
  queue managers that are unavailable for extended periods.
  
  
  Platforms affected:
  MultiPlatform
  
  ****************************************************************
  PROBLEM DESCRIPTION:
  The issue relating to the size of the FDCs is caused by a
  significant quantity of diagnostic information now added to
  every FDC produced by the repository manager process (amqrrmfa).
  In combination with FDCs which are generated frequently because
  of the nature of their state, this can cause rapid growth in the
  disk requirements for the error logging
  
  The problem was caused in this instance by a situation that a
  number of cluster objects were subject to. Specifically, a queue
  manager
  not in contact with the cluster for a significant period of
  time.
  

Problem conclusion

• FDCs should be reserved to document unexpected circumstances
  that occur and nor reporting future events.
  The FDC has now been removed but the AMQ9469 error continues to
  be reported in the logs.
  This will preserve space where users might have many queue
  managers in an unavailable state for extended periods.
  
  ---------------------------------------------------------------
  The fix is targeted for delivery in the following PTFs:
  
  Version    Maintenance Level
  v7.5       7.5.0.9
  v8.0       8.0.0.6
  v9.0 CD    9.0.1
  v9.0 LTS   9.0.0.1
  
  The latest available maintenance can be obtained from
  'WebSphere MQ Recommended Fixes'
  http://www-1.ibm.com/support/docview.wss?rs=171&uid=swg27006037
  
  If the maintenance level is not yet available information on
  its planned availability can be found in 'WebSphere MQ
  Planned Maintenance Release Dates'
  http://www-1.ibm.com/support/docview.wss?rs=171&uid=swg27006309
  ---------------------------------------------------------------
  

Temporary fix

Comments

APAR Information

• APAR is sysrouted FROM one or more of the following:

• APAR is sysrouted TO one or more of the following:

  PI69996 IT36358

Fix information

• Fixed component name

  WMQ BASE MULTIP

• Fixed component ID

  5724H7251

Applicable component levels