SMC protocol support

6.10 LPAR mode z/VM guest KVM guest

The shared memory communication (SMC) protocol is an addition to TCP/IP and can be used transparently for shared memory communications.

The SMC protocol can be used for connections through:
  • Shared Memory Communications through RDMA (SMC-R) with RoCE devices.
  • Shared Memory Communications Direct (SMC-D) with ISM devices.
If both variants are available for a connection, SMC-D is used.

Prerequisites

SMC connections are initiated through TCP/IP. Hence, the communication partners must be able to reach each other through TCP/IP.

An SMC connection requires both communication partners to support SMC. Unless both partners support SMC, the connection falls back to TCP/IP. Similarly, a version 2 SMC connection requires both communication partners to support version 2. If one partner does not support version 2, the connection falls back to version 1.

The SMC-R protocol requires:
  • A system with a RoCE Express adapter.
  • For SMC-Rv1, the communication partners must be in the same subnet. For SMC-Rv2 and using RoCE Express2 or later, communication partners can be in different IP subnets.
The SMC-D protocol requires:
  • A system with an Internal Shared Memory (ISM) device. ISM devices are supported for Linux® in LPAR mode and for Linux on z/VM®.
  • The communication partners must be running on the same CPC.
  • ISM devices must have the same virtual channel ID (VCHID) on both communication partners to be usable for SMC-D communication.
  • For SMC-Dv1, the communication partners must be in the same subnet. As of IBM z15 using SMC-Dv2, communication partners can be in different IP subnets.

To use SMC on Linux, a socket application must use the AF_SMC address family. For AF_SMC support in existing applications without code changes, the SMC-Tools package provides a preload library and the smc_run command. For more information about these tools and how to convert socket applications from AF_INET or AF_INET6 to AF_SMC, see Setting up the SMC support.

Features

The AF_SMC address family provides DMA communication through remote or internal shared memory. Benefits include:
  • Transparency to existing TCP/IP applications with the preload library and smc_run.
  • Low latency
  • Lower CPU usage compared to native TCP/IP