SMC-R switch configuration issues

Remote Direct Memory Access (RDMA) processing requires standard 10 GbE switch support, and distance limitations might exist. Enable the global pause frame (a standard Ethernet switch feature for Ethernet flow control that is described in the IEEE 802.3x standard) on the switch.

When the SMCReason field of the Netstat ALL/-A report is 00005013 - RDMA CONNECTIVITY FAILURE, VTAM® was not able to complete the SMC-R Link Confirm flow, which usually indicates a switch configuration issue. The Link Confirm message is the first data sent over the RDMA over Converged Ethernet (RoCE) fabric. Check for the following issues:

If you are using VLANs, verify that the VLAN configuration on the RoCE switch ports is consistent with the VLAN configuration on the OSD switch ports.

For example, the OSD switch ports might be configured properly, with no VLAN ID or the default VLAN ID, but the RoCE switch ports have a different VLAN ID configured, such as trunk mode with VLAN IDs 400 and 500.

For more information about configuring VLANs with SMC-R, see SMC-R VLAN configuration issues.
Verify that your cable is plugged into the correct port on the IBM® 10 GbE RoCE Express® feature and into the correct port on the switch.

For example, the cable might be plugged into the correct port on the 10 GbE RoCE Express feature but into the wrong port on the switch, or the cable might be plugged into the correct port on the switch but into the wrong port on the 10 GbE RoCE Express feature.
Verify that the MTU value configured on the switch is large enough to support your configured MTU size on GLOBALTCPIP SMCR MTU for this interface.

Enable jumbo frame support on the RoCE switch ports.
Multiple switches are in use but the switch uplinks are not configured properly.
For some switches (for example the IBM RackSwitch G8264), you might need to configure the RoCE switch ports as edge ports. This places the port in the forwarding state as soon as the link is up, reducing delays due to Spanning Tree Protocol processing.

Verify that you have Ethernet flow control enabled on your switch. Ethernet flow control is implemented by using pause frames. If the control is not enabled, this can cause the switch to be overrun leading to packet loss.