PM92899: 5C6-00C510A4 was issued by Queue Manager in CFSTRUCT due to an IXLFORCE call returning with IXLRSNCODEREBUILDINPROGRESS

Obtain the fix for this APAR.

APAR status

• Closed as program error.

Error description

• 5C6-00C510A4 issued by queue manager in CFSTRUCT due to an
  IXLFORCE call returning with IXLRSNCODEREBUILDINPROGRESS
  .
  A structure can only be recovered when none of the
  connections to it is in a "loss of connectivity" state.
  Thus, the long delay between the initial loss of
  connectivity and the adequate processing of this event,
  triggered by the queue managers receiving the
  EEPLXESRECOMMENDACTION  events, creates a rather large
  timing window for this issue to occur.
  As can be seen in the syslog for each of these systems,
  an attempt to initiate a duplexing build of a structure
  was made after the queue managers had lost connectivity,
  but before their connections could be recovered.
  Thus the duplexing build of the structures failed, as
  indicated by the IXC538I message issued in the syslog,
  and the systems went into a wait mode - setting a trigger
  for events on the structure to initiate another duplexing
  rebuild when the structure is in a different
  state.
  Once all the queue managers have received the
  EEPLXESRECOMMENDACTION event for a given structure,
  recoveries, that were pending so far due to remaining
  connections to the queue manager as indicated by CSQE149I
  messeages, start to get processed. As part of the
  processing, two IXLFORCE calls are issued: the first
  one to force all the connections of the structure, and
  the second one to delete the structure itself.
  After the first IXLFORCE call, the structure is in a
  state where a duplexing build can take place, and the
  system goes ahead and initiates the duplexing build
  straight away.
  Thus, the queue manager's second IXLFORCE call to
  delete the structure now fails with
  IXLRSNCODEREBUILDINPROGRESS.
  The 00C510A4 abend is issued by the structure task.
  .
  DUMP TITLE=xxxx,ABN=5C6-00C510A4,U=SYSOPR,C=R3600.710.CFM
  -CSQECONN,M=CSQGFRCV,LOC=CSQELPLM.CSQECONN+00003C90
  

Local fix

Problem summary

• ****************************************************************
  * USERS AFFECTED: All users of WebSphere MQ for z/OS Version 7 *
  *                 Release 1 Modification 0.                    *
  ****************************************************************
  * PROBLEM DESCRIPTION: CF structure recovery fails and the     *
  *                      queue manager abends with 5C6-00C5104A, *
  *                      followed by abend 5C6-00C5104D and      *
  *                      terminates with reason 00C510AB.        *
  ****************************************************************
  * RECOMMENDATION:                                              *
  ****************************************************************
  There are multiple queue managers in a QSG. When the systems
  loose connectivity to the coupling facility, the connectors for
  the different queue managers go into a "loss of connectivity"
  state. If a duplexing build is initiated whilst at least one of
  the connectors is still in that state, it will fail and wait
  for the situation to be resolved.
  As part of recovery processing, two IXLFORCE calls are issued:
  the first one to force all the connections of the structure,
  and the second one to delete the structure itself.
  After the first IXLFORCE call, the structure is in a state
  where a duplexing build can take place, and the system goes
  ahead and initiates the duplexing build straight away. Thus,
  the queue manager's second IXLFORCE call to delete the
  structure now fails with IXLRSNCODEREBUILDINPROGRESS. The
  00C510A4 abend is issued by the structure task.
  

Problem conclusion

• The code was changed to tolerate the IXLRSNCODEREBUILDINPROGRESS
  reason code from the IXLFORCE call. The queue manager will
  attempt it several times before ending the CF structure recovery
  with message CSQE101I, structure in use.
  
  Message CSQE159I was added to indicate that the structure is
  being rebuild by the system and the queue manager goes into a
  wait.
  
  
  The follow message will be added to the info center at
  WebSphere MQ
    > Reference
      > Messages
        > Messages and reason codes for z/OS
          > Messages for WebSphere MQ for z/OS
            > Coupling Facility manager messages (CSQE...)
  
  CSQE149I: csect-name Waiting for other queue managers to
            disconnect from structure structure-name
  
  Explanation
  
  The queue manager has lost connectivity to the coupling
  facility, in which the identified structure is allocated,
  but cannot delete the structure or initiate a system-managed
  rebuild because a structure rebuild is currently in progress.
  
  System action
  
  The queue manager will periodically retry the attempted
  operation until the structure rebuild is finished.
  
  100Y
  101Y
  102Y
  103Y
  104Y
  105Y
  CSQECONN
  CSQFEDIC
  CSQFEDIE
  CSQFEDIF
  CSQFEDIK
  CSQFEDIU
  CSQFMTXC
  CSQFMTXE
  CSQFMTXF
  CSQFMTXK
  CSQFMTXU
  

Temporary fix

• *********
  * HIPER *
  *********
  

Comments

APAR Information

• APAR is sysrouted FROM one or more of the following:

• APAR is sysrouted TO one or more of the following:

  UK96254 UK96255 UK96256 UK96257 UK96258 UK96259

Modules/Macros

• CSQECONN CSQFEDIC CSQFEDIE CSQFEDIF CSQFEDIK
  CSQFEDIU CSQFMTXC CSQFMTXE CSQFMTXF CSQFMTXK CSQFMTXU
  

Fix information

• Fixed component name

  WMQ Z/OS V7

• Fixed component ID

  5655R3600

Applicable component levels

• R100 PSY UK96254

     UP13/09/27 P F309 Ž

• R101 PSY UK96255

     UP13/09/27 P F309 Ž

• R102 PSY UK96256

     UP13/09/27 P F309 Ž

• R103 PSY UK96257

     UP13/09/27 P F309 Ž

• R104 PSY UK96258

     UP13/09/27 P F309 Ž

• R105 PSY UK96259

     UP13/09/27 P F309 Ž

Fix is available

• Select the PTF appropriate for your component level. You will be required to sign in. Distribution on physical media is not available in all countries.