Rendezvous processing

The decision about whether an eligible connection uses SMC communications is reached during traditional TCP connection establishment. Rendezvous processing is the term that is used to describe the sequence of connection management flows (shown in Figure 1) that are required to establish SMC-R communications between two peers.

The exchange of information occurs in different stages:

• Extra information in TCP connection establishment flows
  Applications still use the standard three-way handshake mechanisms to establish TCP connections. When SMC-R or SMC-D communications are enabled, the client and the server can indicate support for either SMC-R or SMC-D protocols:

  • The client adds TCP options settings in the SYN request to indicate which protocols that the client supports.
  • The server responds with TCP options settings in the SYN-ACK response to indicate which protocols that the server supports.

  No additional exchange of information is required in this stage of the rendezvous processing.
• In-band SMC Connection Layer Control (CLC) messages

  Conceptually, CLC messages are similar to the SSL handshake processing that occurs after the TCP connection is established.

  SMC-D protocols are preferred over SMC-R protocols. If both the client and the server indicate support for SMC-D processing, after the TCP connection is established, the client and the server negotiate the use of SMC-D for this TCP connection by using SMC-D CLC messages that flow as in-band data over the TCP connection.

  The SMC-D CLC messages exchange the following information:

  • Peer identification information
  • SMC-D buffer information that is necessary to exchange data between the peers

  For detailed description about the information that SMC-D CLC messages exchange, see SMC-D links.

  If the attempt to establish SMC-D communications succeeds, the SMC-D link is established, and the peers can start exchanging data by using the SMC-D buffers.

  If SMC-D communications is not available, and both the client and the server indicate support for SMC-R processing, the client and the server negotiate the use of SMC-R for this TCP connection by using SMC-R CLC messages that flow as in-band data over the TCP connection.

  The SMC-R CLC messages exchange the following information:

  • Network routing information
  • RDMA over Converged Ethernet (RoCE) routing credentials
  • SMC-R buffer information that is necessary to select or establish the RoCE path between the peers

  For detailed description about the information that SMC-R CLC messages exchange, see SMC-R links.

• SMC-R Link Layer Control (LLC) messages

  After the SMC-R information is exchanged, SMC-R LLC messages are exchanged across the RoCE fabric to confirm that the RoCE information is correct and that the remote memory can be accessed. This stage is skipped if an existing RoCE connection is used for this TCP connection.

The TCP/IP stack does not allow the client and server applications to exchange application data during rendezvous processing. Because no application data is exchanged, the TCP connection can revert to IP protocols if there is a failure during the setup of the SMC communications. However, after the TCP connection is committed to using SMC protocols, the TCP connection cannot fall back to using IP protocols if SMC communications encounter an error.

• For SMC-R communications, the TCP connection is committed to use SMC logic after the RoCE connection is confirmed by using the SMC-R LLC messages.
• For SMC-D communications, the TCP connection is committed to use SMC logic after the successful exchange of SMC-D CLC messages.

Both the client and the server nodes maintain a series of rendezvous timers to ensure that the rendezvous processing completes in time. If one of the timed events does not complete as expected, the TCP connection reverts to using IP protocols. However, future TCP connections can still attempt to use SMC.

To reduce the overhead of persistent rendezvous failures (TCP connections reverting to using IP protocols) to the same destination IP address, you can use the default AUTOCACHE function. This function is controlled by the AUTOCACHE and NOAUTOCACHE subparameters of the SMCGLOBAL parameter on the GLOBALCONFIG profile statement and is set to AUTOCACHE by default. The AUTOCACHE function caches rendezvous failures per IP address destination. If the AUTOCACHE function detects too many rendezvous failures to a specific IP address, the function prevents additional rendezvous attempts to that IP address. The AUTOCACHE function is started only when you enable SMC. For more information about enabling SMC, see the description of the GLOBALCONFIG SMCR and SMCD parameters in z/OS Communications Server: IP Configuration Reference.