RDMA Over Converged Ethernet (RoCE) version 2

Starting with AIX® Version 7.3, Technology Level 2, the AIX operating system supports RoCE version 2 (RoCE v2).

RoCE is a network protocol that enables Remote Direct Memory Access (RDMA) over an Ethernet network. The two versions of the RoCE protocol are RoCE version 1 (RoCE v1) and RoCE v2. RoCE v2 protocol packets can travel across subnets, whereas RoCE v1 protocol packets cannot travel across subnets. RoCE v1 protocol allows communication between hosts that are located in the same Ethernet broadcast domain. RoCE v2 protocol packets carry an Internet Protocol (IP) header that allows traversal of IP Layer 3 (L3) routers. The RoCE v2 protocol packets also carry a User Datagram Protocol (UDP) header that acts as a stateless encapsulation layer for the RDMA Transport Protocol Packets over IP. RoCE v2 protocol is also known as IP Routable RoCE.

Configuration of RoCE protocol

You can use the no command to enable or disable the RoCE v2 protocol mode. RoCE v1 is the default protocol mode.

To enable the RoCE v2 protocol mode, enter the following command:

no -r -o roce_v2_mode=1

To enable the RoCE v1 protocol mode, enter the following command:

no -r -o roce_v2_mode=0

Limitations

The following are the limitations of using the RoCEv2 protocol mode:

The PCIe3 40 GbE RoCE adapter supports only the RoCE v1 protocol mode. When you enable the RoCE v2 protocol mode by using the no command, the RDMA feature is disabled automatically for the PCIe3 40 GbE RoCE adapter.
RoCE v2 protocol mode on AIX does not support Internet Protocol Version 6 (IPv6).
On a single logical partition (LPAR), you can support either RoCE v1 or RoCE v2 protocol mode. For successful communication between networks, both the communicating nodes must be in the same RoCE protocol mode.