Switched fabric topology terminology
Fibre channel is a networking technology primarily used for storage area networking. FICON® itself is a protocol layer created by IBM® which builds upon the fibre channel transport protocol. A switched fabric is a fibre channel topology in which individual node ports are interconnected and managed by switches.
- Director
- An I/O router that can connect any input port to any output port, which may be referred to as a switch.
- Central processor complex (CPC)
- A collection of IBM Z® hardware which is contained in a mainframe (main storage, one or more central processors, and channels), and which may be referred to as a host system, or simply a host.
- Channel (CHPID or PCHID)
- A FICON port on a CPC, which may be referred to as a host port.
- Control unit (CU)
- A storage unit, which may be referred to as a storage system, or simply storage.
- Channel path (path)
- A series of point-to-point connections that form a route through the network between two communicating nodes, which may be referred to as a data link, or simply a link.
- Exchange
- An I/O operation routed through the fabric over a data link.
Figure 1 shows an example of a switched fabric.
_Port,
prefixed by one or more letters to signify the type of connectivity
they provide in the fabric infrastructure. The hardware components
have one or more of the following ports: - N_Ports
- Node ports, which are on a host or storage system.
- F_Ports
- Fabric ports, which are on a switch and provide connections between a node and the network.
- E_Ports
- Expansion ports, which are on a switch and connect to other switches, sometimes by means of inter-switch links (ISLs).
Fabric login is the process by which an N_Port (control unit or channel) registers its presence and establishes the route through the fabric. This occurs during initialization of the channel and the control unit.
An entry port is the F_Port or E_Port where the I/O enters the switch, and an exit port is the F_Port or E_Port where the I/O leaves the switch. Entry and exit ports are relative to the direction of the route (channel to control unit, or control unit to channel) and location within the route. The switch type is also determined by the route through the fabric. If there is only one switch, then the switch type is the only director. If more than one switch, the first entry port is attached to the source director and the last exit port is attached to the destination director. The switches between the source director and the destination director are known as intermediate directors.
Figure 2 shows two switches, each acting as a source director and destination director depending on the direction of the I/O flow.
If the direction of the I/O is from the processor (source) to the CU (destination), port 00 is the entry port to Director A (Source Director), port 01 is the exit port. Port 20 is the entry port to Director B (destination Director) and port 23 is the exit port.
If the direction of the I/O is from the CU (source) to the processor (destination), port 23 is the entry port to Director B and port 20 is the exit port. Port 01 is the entry port to Director A and port 00 is the exit port.
Routing and grouping methods
Fibre channel infrastructure can further be described by the methods used to route I/O requests through the fabric, as well as how multiple links are grouped to form a logical link. The names of the routing and grouping methods described here are vendor-neutral. Different switch vendors have different names for their routing methods (for example, OXID and exchange based routing) and grouping methods (for example, trunks and port channels).
Static routing
When static routing is in effect, a route through the fabric is assigned at fabric login time. If there is a link failure on the assigned route, one of the alternate routes is used. Figure 3 shows an example of static routing.
Figure 3 shows a static path from port 00 to port 02 with three alternate paths. If the link between port 02 and port 11 fails, one of the alternate ports (01, 03, or 04) will be used.
Dynamic routing
When dynamic routing is in effect, each I/O request (exchange) can take a different path through the fabric. If a link failure occurs, any eligible path will be used. The algorithm used to determine how I/O requests are routed through the fabric, as well as how eligible paths are determined, is specific to each implementation and vendor. For example, I/O requests may be distributed across the different paths according to the available bandwidth. Additionally, the set of eligible paths may be determined at fabric build or fabric login time.
Figure 4 shows an example of dynamic routing.
I/O requests, or exchanges, from port 00 can use either port 01, 02, 03, or 04. Exchange 1 (XID 1) uses port 02; exchange 2 (XID 2) uses port 04.
Link aggregation
Aggregation means that multiple ISLs are logically grouped to act as a single, higher bandwidth ISL. If there is a link failure, the affected link is removed. The path is intact as long as one link in the aggregation is active. Aggregation can be used with either static or dynamic routing. The formation of aggregate links and how traffic is distributed between the eligible links is vendor-specific.
Figure 5 shows an example of link aggregation.
In Figure 5, I/O requests are routed to the aggregate consisting of ports 01 and 02 (agg-1). Depending on the vendor specific implementation, this aggregate acts as a single logical link and frames can be routed through either ISL, or a single link can be designated within the aggregate as the active link. If one link or the active link in agg-1 fails, agg-1 will still be used since there is a second functioning link. If both links through agg-1 fail, then the second aggregate consisting of ports 03 and 04 (agg-2) may be used. This example shows an aggregate link used with static routing. However, this is applicable to dynamic routing as well.