IBM Support

PI95380: Queue manager ceases to participate in cluster after REFRESH CLUSTER, mqrc=2189.

A fix is available

Subscribe

You can track all active APARs for this component.

 

APAR status

  • Closed as program error.

Error description

  • In this scenario there were local logger filesystem availability
    problems.  Because of this, there were 1000s of application
    messages on the SYSTEM.CLUSTER.TRANSMIT.QUEUE, and one or more
    channels held indoubt batches of messages.  At this time the
    administrator ran REFRESH CLUSTER on the local queue manager.
    The indoubt batches of messages became rolled back (probably
    automatically due to log space shortage, though the same effect
    could be seen if using RESOLVE CHANNEL to roll back the batch),
    so the messages appeared on the transmission queue again.
    
    After this the local queue manager's repository manager program
    tried to reallocate messages from the rolled-back indoubt
    batches.  The reallocation routine found that the cluster cache
    no longer knew anything about the queues for which they were
    destined.  The repository manager did not break from its
    reallocation routine to ask the full repositories for details of
    the queues, so it suffered repeated
    MQRC_CLUSTER_RESOLUTION_ERROR errors over a period of many
    minutes, which were visible in the MQ trace for the
    rrmReallocMsgs process.
    
    Application calls to MQOPEN for queues not known locally will
    fail with 2189 MQRC_CLUSTER_RESOLUTION_ERROR.
    Other symptoms include:
    -- a failure to recognize other queue managers in the
       cluster including the QMGR hosting the cluster Q.
    -- message buildup on the SCCQ and the SCTQ.
    

Local fix

Problem summary

  • ****************************************************************
    * USERS AFFECTED: All users of IBM MQ for z/OS Version 8       *
    *                 Release 0 Modification 0.                    *
    ****************************************************************
    * PROBLEM DESCRIPTION: A queue manager experiencing problems   *
    *                      while there are inflight batches of     *
    *                      messages may cease to participate in a  *
    *                      cluster after a REFRESH CLUSTER         *
    *                      command. Multiple MQPUT1 failures with  *
    *                      reason 88D are seen in chinit trace,    *
    *                      RC 2189, MQRC_CLUSTER_RESOLUTION_ERROR. *
    ****************************************************************
    The reallocation routine within the repository manager was
    suffering a MQRC_CLUSTER_RESOLUTION_ERROR condition repeatedly,
    for the same message each time.  There were 1000s of messages on
    the cluster transmission queue, and because of a flaw in the
    logic flow it would re-read the same message 1000s of times.
    It did not break from this loop to send the query to the full
    repository that was necessary to relieve this situation.
    

Problem conclusion

  • Logic has been changed such that the reallocation routine
    schedules a re-run of itself in 60 seconds time, and breaks from
    its work to allow the repository manager to request information
    from the full repositories.
    

Temporary fix

Comments

APAR Information

  • APAR number

    PI95380

  • Reported component name

    IBM MQ Z/OS V8

  • Reported component ID

    5655W9700

  • Reported release

    000

  • Status

    CLOSED PER

  • PE

    NoPE

  • HIPER

    NoHIPER

  • Special Attention

    NoSpecatt / Xsystem

  • Submitted date

    2018-03-19

  • Closed date

    2020-08-10

  • Last modified date

    2020-11-02

  • APAR is sysrouted FROM one or more of the following:

    IT24269

  • APAR is sysrouted TO one or more of the following:

    PH28037 UI70970

Modules/Macros

  • CSQXRRMF
    

Fix information

  • Fixed component name

    IBM MQ Z/OS V8

  • Fixed component ID

    5655W9700

Applicable component levels

  • R000 PSY UI70970

       UP20/10/13 P F010

Fix is available

  • Select the PTF appropriate for your component level. You will be required to sign in. Distribution on physical media is not available in all countries.

[{"Line of Business":{"code":"LOB36","label":"IBM Automation"},"Business Unit":{"code":"BU053","label":"Cloud & Data Platform"},"Product":{"code":"SSYHRD","label":"IBM MQ"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"8.0"}]

Document Information

Modified date:
14 December 2020