OA55243: V XCF,OFFLINE ABEND026 RSN0811001 IXL102A IXL109I FENCE TIMED OUT

APAR status

• Closed as fixed if next.

Error description

• When multiple systems are partitioned from the sysplex in a
  close timeframe, there maybe a delay in partitioning of some
  systems due to serialization contention on the RECOVERY record
  on the SYSPLEX CDS. Fencing timeouts are reported by IXC109I yet
  
  we can see that the fencing operation actually completed
  successfully as indicated in a SYMREC cut by IXCS2WTO. We also
  see the operator prompted with an unnecessary IXC102A.
  
  IXC109I FENCE OF SYSTEM MXB3 TIMED OUT.
  IXC109I FENCE OF SYSTEM MXD1 TIMED OUT.
  IXC109I FENCE OF SYSTEM MXD2 TIMED OUT.
  IXC109I FENCE OF SYSTEM MXD3 TIMED OUT.
  5211 IXC102A XCF IS WAITING FOR SYSTEM MXD1 DEACTIVATION. REPLY
  DOWN WHEN MVS ON  MXD1 HAS BEEN SYSTEM RESET
  
  5208 IXC102A XCF IS WAITING FOR SYSTEM MXD3 DEACTIVATION. REPLY
  DOWN WHEN MVS ON MXD3 HAS BEEN SYSTEM RESET
  
  5209 IXC102A XCF IS WAITING FOR SYSTEM MXD2 DEACTIVATION. REPLY
  DOWN WHEN MVS ON  MXD2 HAS BEEN SYSTEM RESET
  
  5210 IXC102A XCF IS WAITING FOR SYSTEM MXB3 DEACTIVATION. REPLY
  DOWN WHEN MVS ON MXB3 HAS BEEN SYSTEM RESET
  
  VERIFICATION STEPS:
  1. Verify in the log for message IXL109I indicating 'FENCE OF
  SYSTEM XXXX TIMED OUT'
  2. Check for a SYMREC record in logrec prior to this indicating
  the Fence/Isolate was success. The SYMREC record will show:
  DESCRIPTION OF FUNCTION:  SYSXCF SFM FST
  PROBLEM ID:               FENCING
  
  3. Check the FST control block logged in the SYMREC record for
  LFSSRC = 00000000 at +x'34'.  Here is an example of an FST
  logged:
  KEY = FF00     LENGTH = 000528 (0210)
  +000    00000001    00000000    00000000    00000000
  +010    D4E7C3F2    40404040    00000002    D4E7C4F1
  +020    40404040    07000516    00000000    CF772964
  +030    03000516    00000000    00000020    00000000
  +040    00000020    00000000    00000000    00000000
  

Local fix

Problem summary

• ****************************************************************
  * USERS AFFECTED: Users running in a z/OS parallel sysplex     *
  *                 configuration.                               *
  ****************************************************************
  * PROBLEM DESCRIPTION: Completion of XCF sysplex system        *
  *                      partitioning for demised systems is     *
  *                      delayed due to contention on sysplex    *
  *                      couple data set resources held by       *
  *                      systems being partitioned.              *
  ****************************************************************
  * RECOMMENDATION:                                              *
  ****************************************************************
  XCF VARY operator commands initiated the partitioning of
  multiple sysplex systems simultaneously. Sysplex system
  partitioning completion for at least one system was delayed by
  many minutes due to sysplex couple data set resources (sysplex
  couple data set RECOVERY record) still being held by many of
  the partitioned systems.
  
  Even though I/O isolation (fencing) of the targeted system(s)
  completed, partitioning processing couldn't proceed with
  complete partition clean up due to the delay in the releasing
  of needed resources. MSGIXC109I was issued indicating fencing
  for some of the partitioned systems TIMED OUT even though
  fencing completed many minutes prior.  Eventually, the resources
  under contention were recovered, and the partition monitoring
  system obtained the needed resources to complete the
  partitioning process.
  
  This APAR is being closed FIN with concurrence from the
  submitting customer. A solution to this problem will be
  delivered in a z/OS release (if any) to be available within
  the next 36 months.
  

Problem conclusion

Temporary fix

Comments

• This APAR is being closed FIN with concurrence from the
  submitting customer. A solution to this problem will be
  delivered in a z/OS release (if any) to be available within
  the next 36 months.
  

APAR Information

• APAR is sysrouted FROM one or more of the following:

• APAR is sysrouted TO one or more of the following:

Fix information

Applicable component levels