Steps for diagnosing Tier 1 non-z/OS sysplex distribution problems
For an in-depth explanation of distribution to non-z/OS® targets, see the Sysplex distribution with DataPower® section in z/OS Communications Server: IP Configuration Guide.
Tip: Tier 1 non-z/OS sysplex distribution is available for both IPv4 and IPv6. The following examples describe how to diagnose IPv4 distribution problems with DataPower, but this function applies to
IPv6 as well.
- For Tier 1 sysplex distribution
- On the tier 1 distributor, run the Netstat VIPADCFG/-F DETAIL
report to see the tier 1 configuration statement information.
- In the VIPA DEFINE section of the report, verify the new flag field, 1, which indicates that this is a VIPA definition for a tier 1 DVIPA.
- In the VIPA DISTRIBUTE section, verify the tier 1 configuration
parameters.
- The flag fields show that the targets are Tier1, and that the distribution method is target controlled (TargCtrl). Target controlled distribution means that the tier 1 targets will send weight recommendations to the distributor. The other routing types supported are ROUNDROBIN and WEIGHTEDACTIVE.
- The routing type used to send packets to the non-z/OS tier 1 targets is GRE. GRE is the only type that is supported.
- When the RtgType is GRE, a control connection is established from
the distributor to each target. In this example, the destination
port (CtrlPort) that will be used for the control connection is 1702.
The control connection will be used to receive:
- Weight recommendations from the target (since the distribution method is TargCtrl) - the distributor will load balance across the tier 1 targets using these weights similar to how the distributor uses WLM weight recommendations to load balance across z/OS targets
- Connection awareness information (since the routing type is GRE) - this information will indicate if the target has a ready server listening on port 9000, and will also be used to indicate when connections are established and terminated.
- The group name is CICSGROUP. The group name is used when a tier
2 sysplex distributor is being used to correlate tier 1 targets with
their tier 2 targets. This is explained further in the following tier
2 distribution section.
Dynamic VIPA Information: VIPA Define: IpAddr/PrefixLen: 9.42.130.251/24 Moveable: Immediate SrvMgr: No Flg: 1 VIPA Distribute: Dest: 9.42.130.251..9000 DestXCF: 9.42.105.53 SysPt: No TimAff: No Flg: TargCtrl Tier1 OptLoc: No GrpName: CICSGROUP RtgType: GRE CtrlPort: 01702 Dest: 9.42.130.251..9000 DestXCF: 9.42.105.73 SysPt: No TimAff: No Flg: TargCtrl Tier1 OptLoc: No GrpName: CICSGROUP RtgType: GRE CtrlPort: 01702 Dest: 9.42.130.251..9000 DestXCF: 9.42.103.215 SysPt: No TimAff: No Flg: TargCtrl Tier1 OptLoc: No GrpName: CICSGROUP RtgType: GRE CtrlPort: 01702 Dest: 9.42.130.251..9000 DestXCF: 9.42.103.216 SysPt: No TimAff: No Flg: TargCtrl Tier1 OptLoc: No GrpName: CICSGROUP RtgType: GRE CtrlPort: 01702Run the Netstat ALLCONN/-a report to verify that a control connection is active to each non-z/OS target. The foreign socket destination IP address and destination port correspond to the DestXCF IP address (of the non-z/OS target) and the control port of 1702 from the VIPA distribute section. In this example report, there are four TCP connections in established state to each of the configured Tier 1 targets.USER1@VIC018:/:> netstat -a MVS TCP/IP NETSTAT CS V1R11 TCPIP Name: TCPCS 17:23:36 User Id Conn State ------- ---- ----- TCPCS 00000014 Establsh Local Socket: 9.42.104.4..1025 Foreign Socket: 9.42.103.216..1702 TCPCS 00000018 Establsh Local Socket: 9.42.104.4..1027 Foreign Socket: 9.42.105.73..1702 TCPCS 00000019 Establsh Local Socket: 9.42.104.4..1028 Foreign Socket: 9.42.105.53..1702 TCPCS 00000017 Establsh Local Socket: 9.42.104.4..1026 Foreign Socket: 9.42.103.215..1702Run the Netstat VDPT/-O report to see the current status of the Tier 1 targets. Verify the weight recommendations are non-zero and that there is a ready server for each Tier 1 target. This information is updated each polling interval.- The T1Wt field is the weight recommendation reported by each target. In the example, all are non-zero.
- The ready count is 1 for each Tier 1 target indicating that each has a server Listening on port 9000.
- The CWt field is a CPC weight. This field should be non-zero if Tier 2 distributors are configured. If there are no corresponding Tier 2 distribution statements, then this field will be zero. This is explained further in the following Tier 2 distribution section.
Dynamic VIPA Destination Port Table for non-z/OS targets: Dest: 9.42.130.251..9000 Target Addr: 9.42.103.215 TotalConn: 0000000000 Rdy: 001 Wt: 03 CWt: 032 Flg: TargCtrl T1Wt: 420 ActConn: 0000000000 Dest: 9.42.130.251..9000 Target Addr: 9.42.103.216 TotalConn: 0000000000 Rdy: 001 Wt: 05 CWt: 032 Flg: TargCtrl T1Wt: 672 ActConn: 0000000000 Dest: 9.42.130.251..9000 Target Addr: 9.42.105.53 TotalConn: 0000000000 Rdy: 001 Wt: 03 CWt: 064 Flg: TargCtrl T1Wt: 420 ActConn: 0000000000 Dest: 9.42.130.251..9000 Target Addr: 9.42.105.73 TotalConn: 0000000000 Rdy: 001 Wt: 03 CWt: 064 Flg: TargCtrl T1Wt: 415 ActConn: 0000000000
- Tier 2 sysplex distribution
- A Tier 1 target may distribute connections to Tier 2 targets on the same CPC using a Tier 2 sysplex distributor configured with CPCSCOPE. The Tier 2 sysplex distributor will only load balance to z/OS targets that are on the same CPC.
- In the VIPA DEFINE section of the report, the new flag field, 2, indicates that this is a VIPA definition for a Tier 2 DVIPA. The flag, "C", indicates that the Tier 2 DVIPA is configured with CPCSCOPE.
- Each Tier 2 distributor will send a combined CPC weight of all ready servers to the Tier 1 distributor; the group name is used to correlate a group of Tier 1 targets with their Tier 2 targets.
- On each Tier 2 distributor run the Netstat VIPADCFG/-F DETAIL report to verify that the same group name is used on the Tier 1 and corresponding Tier 2 VIPADISTRIBUTE statements. In the example report from the VIPA distribute section, the group name "CICSGROUP" matches the Tier 1 group name.
- A combined weight will be used to update a Tier 1 target with
a matching group name if this Tier 2 distributor (and therefore its
Tier 2 targets) are on the CPC used by that Tier 1 target. A VIPADEFINE
CPCSCOPE definition is required to determine the CPC used by a Tier
1 target. If the IP address of the target is in the same subnet of
this DVIPA, then its targets are on that CPC.
- In the example report from the VIPA define section, DVIPA 9.42.103.17 is a CPCSCOPE DVIPA because the flag field is "C".
- The subnet for this DVIPA is 9.42.103.xx/24. So all Tier 1 targets (from the Tier 1 Netstat VDPT/-O report above) in this subnet are using the same CPC as that stack.
- The Tier 1 targets in this subnet are 9.42.103.215 and 9.42.103.216. So as combined weights are received from a Tier 2 distributor, with the same CPC ID as the stack in which the CPCSCOPE definition was defined, the Tier 1 combined weight fields (CWt) for those targets (in the CPCSCOPE subnet) are updated with those weights.
- In this example, the CPCSCOPE DVIPA is configured on the same stack as the Tier 2 VIPADISTRIBUTE statement. So this Tier 2 distributor will be providing the combined weights for the non-z/OS targets in the CPCSCOPE DVIPA subnet, 9.42.103.215 and 9.42.103.216. Ideally the CPCSCOPE DVIPA would be defined on the same stack as the Tier 2 Distributor, and VIPABACKUP statements for these DVIPAs would be on the same stacks with the same ranks. However the CPCSCOPE DVIPA could be defined on any stack as long as it is in the same CPC as the Tier 2 distributor.
VIPA Define: IpAddr/PrefixLen: 9.42.103.17/24 Moveable: Immediate SrvMgr: No Flg: C IpAddr/PrefixLen: 203.2.108.3/24 Moveable: Immediate SrvMgr: No Flg: 2C VIPA Distribute: Dest: 203.2.181.3..11000 DestXCF: ALL SysPt: No TimAff: No Flg: ServerWLM Tier2 OptLoc: No GrpName: CICSGROUP - On each Tier 2 distributor, run the Netstat VDPT/-O DETAIL report to see the status of each Tier 2 target. In the example, there is a non-zero ready count of 1 for each server, and there is a non-zero normalized server-specific weight for each target.
- The combined weight in the Tier 1 VDPT (CWt) displays for the
Tier 1 targets on that CPC, 9.42.103.214 and 9.42.103.216 is the total
weight of the Tier 2 ready servers, 32.
Dynamic VIPA Destination Port Table for TCP/IP stacks: Dest: 203.2.108.3..11000 DestXCF: 193.1.1.108 TotalConn: 0000000000 Rdy: 001 WLM: 16 TSR: 100 Flg: ServerWLM, Tier2 TCSR: 100 CER: 100 SEF: 100 Weight: 64 Raw CP: 64 zAAP: 00 zIIP: 00 Proportional CP: 63 zAAP: 00 zIIP: 00 Abnorm: 0000 Health: 100 ActConn: 0000000000 QosPlcAct: *DEFAULT* W/Q: 16 Dest: 203.2.108.3..11000 DestXCF: 193.1.1.181 TotalConn: 0000000000 Rdy: 001 WLM: 16 TSR: 100 Flg: ServerWLM, Tier2 TCSR: 100 CER: 100 SEF: 100 Weight: 64 Raw CP: 64 zAAP: 00 zIIP: 00 Proportional CP: 63 zAAP: 00 zIIP: 00 Abnorm: 0000 Health: 100 ActConn: 0000000000 QosPlcAct: *DEFAULT* W/Q: 16