When configuring devices with multiple paths to the same CPC, select any of the channel paths
or PCIe functions shown in
Figure 1,
Figure 2,
Figure 3, or
Figure 4 that:
- Are available on the CPC you are defining
- Are the correct type (ESCON, FICON®, and so on) to meet the control unit, coupling facility, or network
attachment requirements
- Satisfy the rules regarding the mixing of channel types to a control unit.
However, for maximum availability of the device, OSA network, coupling facility, or PCIe function
on a 3932, 3931, 8562, 8561,
3907, 3906,
2965, 2964, 2828, 2827, 2818, or
2817 model, consider the following guidelines.
- Choose channels or PCIe functions plugged in different I/O domains (see
CU 1). An I/O domain in an I/O cage or I/O drawer contains four I/O cards controlled by a single IFB-MP card. For example, the domain for the IFB-MP card in
H105 in the I/O cage contains slots 01, 03, 06, and 08 (see
Figure 3) and in D109 in the I/O drawer contains slots 02, 05, 08, and 10 (see
Figure 4).
An I/O domain in a PCIe I/O drawer contains eight I/O cards
controlled by a single PCIe-IN card. For example, the domain for the PCIe-IN card in slot 05 in the
38-slot PCIe I/O drawer contains slots 1-4 and 5-9 (see Figure 1) and in
slot 06 in the 20-slot PCIe I/O drawer contains slots 2-5 and 7-10 (see Figure 2).
Note: This action is also recommended for optimum performance of your most heavily used I/O
devices.
When choosing the I/O domains to use, whether from different cages/drawers or the
same cage/drawer, consider using a combination of I/O domains from different books, or
CPC drawers. When you must use InfiniBand, Integrated Coupling Adapter (ICA SR), or PCIe links from the
same book or CPC drawer, try to use InfiniBand, ICA, or PCIe links from different Host Channel Adapter (HCA), ICA SR, or PCIe fanout cards on that book. Refer to your PCHID Report to determine which
InfiniBand, ICA, and PCIe links belong to which fanout cards on a book. If you have multiple paths to the
device and multiple domains available that have the correct channel type, spread the paths across as
many books and CPC drawers and fanout cards as possible.
Redundant I/O Interconnect is a function that allows one IFB-MP or PCIe-IN to
back up another IFB-MP or PCIe-IN for a failure or repair. The IFB-MPs in slot 05
of the I/O cage back up each other (see Figure 3). The
same is true for IFB-MPs in slots 14, 23, and 28. In the I/O drawer, the IFB-MP cards in slot 09
back up each other (see Figure 4). The PCIe-IN cards in slots 05 and 34 of the 38-slot PCIe I/O drawer
back up each other (see Figure 1). The
same is true for the PCIe-IN cards in slots 15 and 24.
Also, the PCIe-IN cards in slots 06 and 16 of the 20-slot PCIe I/O drawer back up
each other (see Figure 2).
Therefore, in the event an InfiniBand
or PCIe cable, fanout card, or book fails, the remaining IFB-MP or
PCIe-IN card controls both domains. Certain failures (for example, the IFB-MP or
PCIe-IN card) might prevent the redundant takeover, which is a reason to spread your paths over
multiple domains.
When configuring Coupling over InfiniBand (CIB) channel paths for the same target CPC or
coupling facility, use InfiniBand links that originate from different books or CPC
drawers and different HCA fanout cards on those books/drawers. Doing so
eliminates the HCA fanout card and the book as a single point of failure where all connectivity
would be lost.
When configuring Coupling Short Reach Gen5 channels
(Integrated Coupling Adapter (ICA SR) channel paths for the same target CPC or coupling facility,
use ICA links that originate from different CPC drawers and different ICA fanout cards on those
drawers. This eliminates the ICA fanout card and the drawer as a single point of failure where all
connectivity would be lost.
When configuring Coupling Express LR (CE LR) channel
paths for the same target CPC or coupling facility, use CE LR links that originate from different
I/O drawers and different PCIe-IN fanout cards on those drawers. This eliminates the PCIe-IN fanout
card and the drawer as a single point of failure where all connectivity would be lost.
- If you define multiple paths from the same InfiniBand, ICA, or PCIe link, distribute paths across different I/O cards (see CU 2).
Also, if you define multiple coupling facility channels to the same coupling facility or to the same
z/OS image, distribute paths across different coupling facility fanout cards, CE
LR cards, ISC mother cards, or ISC daughter cards.
- The following channel path and function types are associated with resource groups and
managed by the CPC: CL5 channel paths and ROCE, ROC2, and ZEDC functions. These channel paths and
function types are associated with the resource groups shown in Table 1,
Table 2 , and
Table 3. The
same resource groups apply to all I/O drawers for the CPC. A resource group can become unavailable
(for example, during an LMC patch install) and all of its channel paths and functions will become
unavailable. When configuring CL5 channel paths for the same target CPC or coupling facility, use CE
LR links that originate from different resource groups. When configuring these function adapter
types, be aware that those which are in the same resource group can all become unavailable at the
same time.
Note: The examples shown in
Figure 1,
Figure 2,
Figure 3 and
Figure 4 merely try to
illustrate availability guidelines and do not represent overall optimal configurations.
The 3932, 3931, 8562, 8561, 3907,
3906,
2965, 2964, 2828, 2827, 2818, and
2817 CPCs have eliminated the concept of FICON channels
or channel cards having affinity to specific SAPs. Now each SAP handles FICON work more on an on-demand basis. That is, as FICON work for any channel arrives, the next available SAP handles that
request. It does not matter if it is an outbound request or an inbound interrupt, the next available
SAP handles the FICON work. There is no requirement to
configure for "SAP affinity," because there is no affinity for FICON. For the other channel types (ESCON, and so on),
the 2828, 2827, 2818, and 2817 CPCs automatically balance
installed channel cards across all available SAPs. The CPC tries to assign an equal number of each
channel card type to each available SAP. All channels on a given I/O card are always in the same
SAP, but it is not predictable which I/O cards are assigned to which SAPs. There are two exceptions
to the previous statements. First, Host Channel Adapters (HCAs) used for coupling are always given
affinity to a SAP on the local book. Second, if an OSA channel is defined as OSD, OSM, or OSX or a
FICON channel is defined as FCP, these channels use QDIO
architecture and, therefore, do not actually use any SAP resource during normal operations. For all
channel types simply follow the preceding recommendations for configuring for RAS, and the SAPs
handle the workload appropriately.