Implementing server and client communications with the endpoint interface (Linux only)

If you develop applications that use the endpoint interface, fewer steps are required for Remote Direct Memory Access (RDMA) connections between server and client than an equivalent process that uses the verbs interface. An example is provided to guide developers through the process on both the server and client sides of the connection.

Note: The RDMA implementation is deprecated and will likely be removed in a future release of IBM® SDK, Java™ Technology Edition, Version 8. A possible alternative is the open source Libfabric library.

The endpoint API is simpler than the verbs API. You can create an endpoint and a connection in a similar way to TCP socket communications, by using bind, connect, and disconnect operations. However, after they are connected, endpoints register memory, post send and receive requests, and unregister memory in a similar way to RDMA connection IDs.

Endpoints belong to an endpoint group; endpoint groups contain endpoints that share common properties or infrastructure. For example, basic endpoint groups can define the following processes:
Connection management
Common event processing and dispatching for endpoints.
RDMA resources
Common associations between RDMA resources, such as queue pairs and completion queues.
RDMA operations
Common requirements for how RDMA resources are used.
You must choose the type of group that meets the requirement for an application. Alternatively you can customize and extend a group to meet specific requirements.
An extension of the basic endpoint group is defined by the RdmaActiveEndpointGroup class. The active endpoint group creates the following threads:
  • A thread for processing connection management events on an event channel
  • A thread for processing completion events on a completion channel

An endpoint group shares the same completion queue, which can hold up to 500 work completion events. If the limit is reached, accepting new connections or creating new endpoints in the endpoint group results in an IOException exception.

The following diagram shows an example of a server and client implementation that uses the endpoint API.
The diagram shows an example of the programming steps that are required for RDMA communication when you develop applications with the endpoint API. Steps are required in a specific order on both the client and server sides of the connection. For a complete description of the communication flow, read the associated text.

The following sections provide an explanation of the server and client implementation that is illustrated in the diagram.

Server flow

The following steps take place on the server side to implement the RDMA communication flow by using the RdmaActiveEndpointGroup() method:
  1. The RDMA endpoint group is created by using the RdmaActiveEndpointGroup() method.
  2. The RDMA server endpoint is created by using the RdmaActiveEndpointGroup.createServerEndpoint() method.
  3. A bind takes place.
  4. The connection is accepted.
  5. For each client request, the following steps take place:
    • The client RDMA endpoint connection is opened.
    • A send or receive work request is posted. The completion of the post is notified to an endpoint by calling the dispatchCQEvent() method.
    • The connection with the client RDMA endpoint is closed.
  6. The RDMA server endpoint is closed.
  7. The RDMA endpoint group is closed.

Client flow

The following steps take place on the client side to implement the RDMA communication flow:
  1. The RDMA endpoint group is created by using the RdmaActiveEndpointGroup() method.
  2. The RDMA client endpoint is created by using the RdmaActiveEndpointGroup.createEndpoint() method.
  3. A connect request is made to the server.
  4. A send or receive work request is posted.
  5. The RDMA client endpoint is closed.
  6. The RDMA endpoint group is closed.