SMC protocol support

Red Hat Enterprise Linux 9.2 LPAR mode z/VM guest KVM guest

The shared memory communication (SMC) protocol is an addition to TCP/IP and can be used transparently for shared memory communications.

The SMC protocol can be used for connections through:
  • Shared Memory Communications through RDMA (SMC-R) with RoCE devices.
  • Shared Memory Communications Direct (SMC-D) with ISM devices
If both variants are available for a connection, SMC-D is used.

Prerequisites

SMC connections are initiated through TCP/IP. Hence, the communication partners must be able to reach each other through TCP/IP.

An SMC connection requires both communication partners to support SMC. Unless both partners support SMC, the connection falls back to TCP/IP.

Similarly, a version 2 SMC connection requires both communication partners to support version 2. If one partner does not support version 2, the connection falls back to version 1.
The SMC-R protocol requires:
  • A system with a RoCE Express adapter.
  • For SMC-R version 1, the communication partners must be in the same subnet. As of RoCE Express2 using SMC-R version 2, communication partners can be in different IP subnets.
The SMC-D protocol requires:
  • A system with an Internal Shared Memory (ISM) device. For more information about ISM devices, see Internal shared memory device driver. ISM devices are supported for Linux in LPAR mode and for Linux on z/VM.
  • The communication partners must be running on the same CPC.
  • ISM devices must have the same virtual channel ID (VCHID) on both peers to be usable for SMC-D version 2 communication.
  • For SMC-D version 1, the communication partners must be in the same subnet. As of IBM z15 using SMC-D version 2, communication partners can be in different IP subnets.

To use SMC on Linux, a socket application must use the AF_SMC address family. For AF_SMC support in existing applications without code changes, the SMC-Tools package provides a preload library and the smc_run command. For more information about these tools and how to convert socket applications from AF_INET or AF_INET6 to AF_SMC, see Setting up the SMC support.

Features

The AF_SMC address family provides DMA communication through remote or internal shared memory. Benefits include:
  • Transparency to existing TCP/IP applications with the preload library and smc_run.
  • Low latency
  • Lower CPU usage compared to native TCP/IP