About this publication

Edit online

This publication provides information that can be used to plan for IBM Fiber Optic Links. This includes coupling facility links, Open Systems Adapter (OSA), zHyperLink Express, and Fibre Connection (FICON®) channels, which is IBM's implementation of the ANSI Fibre Channel Standard with additional support for multimode fibers. Although it contains general information relating to fiber optic cables, components, and optical fiber data processing environments, it includes only what is supported for these fiber optic links.

The Open Systems Adapter-Express (OSA-Express) Ethernet features conform to industry standards as defined herein. Contact your cable provider for further assistance. Refer also to Table 2. 25GbE Networking cards require a technology known as forward-error-correction (FEC) to function. There are two modes, Fire-Code and Reed-Solomon. Reed-Solomon is the IEEE required mode for IBM zSystems 25GbE OSA-Express and 25GbE RoCE cards.
Note: 25GbE networking cards require a technology known as forward-error-correction (FEC) to function. There are two modes, Fire-Code and Reed-Solomon. Reed-Solomon is the IEEE required mode for 25GbE OSA-Express and 25GbE RoCE cards. When connecting 25GbE OSA-Express7S and 25GbE RoCE cards to a Cisco switch, clients must manually set the switch to use FEC RS-IEEE to ensure the ports come online with the correct Reed-Solomon FEC mode.

When connecting 25GbE OSA-Express7S and 25GbE RoCE cards to a Juniper switch, the client must manually set the switch to use gigether-options fec fec108 to ensure the ports come online with the correct Reed-Solomon FEC mode.

  • 25 Gigabit Ethernet (25GBASE-LR, 25GBASE-SR)
    • IEEE 802.3cc
    • IEEE 802.3by
    • SFF-8472 Rev. 12.2
    • SFF-8432
    • SFF-8431 Rev. 3.0
    • SFF-8431 Rev. 4.1
    • SFF-8402
  • 10 Gigabit Ethernet (10GBASE-LR, 10GBASE-SR)
    • IEEE 802.3ae
    • IEEE 802.1q
    • IEEE 802.3x - flow control
    • DIX Version 2 (DIX V2)
  • Gigabit Ethernet (1GBASE-LX, 1GBASE-SX)
    • IEEE 802.3ac
    • IEEE 802.1q
    • IEEE 802.3x - flow control
    • IEEE 802.3z
    • DIX Version 2 (DIX V2)
  • Ethernet (1000BASE-T)
    • IEEE 802.1p
    • IEEE 802.1q
    • IEEE 802.3ab
    • IEEE 802.3ac
    • IEEE 802.3u
    • IEEE 802.3x - flow control
    • DIX Version 2 (DIX V2)

Fibre Connections (FICON):

The Fibre Connections (FICON) features conform to industry standards as defined herein. Contact your cable provider for further assistance. Refer also to Table 2.
  • Fibre Connections (FICON) features (16G-SX, 16G-LX, 32G-SX, 32G-LX)
    • FC-PI-5 Rev. 6.00
    • FC-PI-5 Rev. 6.10
    • SFF-8472 Rev. 11.0
    • SFF-8081
    • SFF-8432
    • SFF-8431 Rev. 4.1

zHyperLink Express:

The zHyperLink Express features conform to industry standards as defined herein. Contact your cable provider for further assistance. Refer also to Table 2.
  • zHyperLink Express (CXP10)
    • IBTA CXP Specification
    • IEEE 802.3ba 100GBASE-SR10 and CPPI electrical interfaces

10GbE RoCE Express:

The 10GbE RoCE Express feature conforms to industry standards as defined herein. Contact your cable provider for further assistance. Refer also to Table 2.
  • RoCE uses the InfiniBand Trade Association-defined (IBTA-defined) transport headers and invariant (end-to-end) cyclical redundancy checking (CRC) (adapter-to-adapter) to protect the InfiniBand transport payload. Refer to the supplement to the InfiniBand Architecture Specification Volume 1 Release 1.2.1, Annex A16 for details.
  • RoCE uses the IEEE-assigned EtherType of 0x8915.
  • The 10 GbE-capable switch must have Pause frame enabled as defined by the IEEE 802.3x standard.

Remote Direct Memory Access (RDMA) over Converged Ethernet (RoCE) is part of the InfiniBand Architecture Specification that provides InfiniBand transport over Ethernet fabrics. It encapsulates InfiniBand transport headers into Ethernet frames using an IEEE-assigned ethertype. One of the key InfiniBand transport mechanisms is RDMA, which is designed to allow transfer of data to or from memory on a remote system with low-latency, high-throughput, and low CPU utilization

Traditional Ethernet transports such as TCP/IP typically use software-based mechanisms for error detection and recovery and are based on the under-lying Ethernet fabric using "best-effort" policy. With the traditional policy the switches typically discard packets in the event of congestion and rely on the upper level transport for packet re-transmission. RoCE, however, uses hardware-based error detection and recovery mechanisms defined by the InfiniBand specification. A RoCE transport performs best when the under-lying Ethernet fabric provides a loss-less capability, where packets are not routinely dropped. This can be accomplished by using DEthernet flow control whereby Global Pause frames are enabled for both transmission and reception on each of the Ethernet switches in the path between the 10GbE RoCE Express features. This capability is enabled by default in the 10GbE RoCE Express feature.

Notes:
  1. When the term FICON/FCP is used, it refers to FICON Express (LX and SX), FICON Express2 (LX and SX), FICON Express4 (LX and SX), FICON Express8 (LX and SX) and FICON Express8S (LX and SX) channels.
  2. When the term OSA is used, it refers to OSA-Express, OSA-Express2 , OSA-Express3, OSA-Express4S, OSA- Express5S, OSA- Express6S, and OSA- Express7S features.
  3. This publication, in conjunction with the publication Maintenance Information for Fiber Optic Links replaces the publication IBM 3044 Fiber-Optic Channel Extender Link Models C02 and D02: Fiber-Optic Cable Planning, Installation, and Maintenance Guide, GC22-7130, and makes it obsolete.
  4. This publication replaces Cabling System Optical Fiber Planning and Installation Guide, GA27-3943.
  5. This publication replaces Fibre channel Connection (FICON) I/O Physical Layer, SA24-7172, and Coupling Links I/O Physical Layer, SA23-0395.