Enabling Java Sockets over Remote Direct Memory Access (JSOR) (Linux only)

JSOR is enabled by specifying a command line system property when you start your Java™ application. A valid configuration file must exist on both the Java application server and client systems, which specifies the RDMA rules for connection. If RDMA services are not available, you can optionally configure your server to receive TCP connections for plain sockets.

Before you begin

Note: The RDMA implementation is deprecated and will likely be removed in a future release of IBM® SDK, Java Technology Edition, Version 8. A possible alternative is the open source Libfabric library.
Read the overview section about Java Sockets over Remote Direct Memory Access (JSOR) (Linux only), which contains some important considerations. In particular, you should note the following points:
  • This feature is available on the following platforms and Remote Direct Memory Access (RDMA) adapters:
    • Linux® 32-bit x-86 with either InfiniBand (IB) or RDMA over Converged Ethernet (RoCE)
    • Linux AMD64/EM64T with IB or RoCE
    • Linux 32-bit POWER® with RoCE
    • Linux 64-bit POWER with RoCE
    • Start of changes for service refresh 2Linux 64-bit POWER8® (Little Endian) with IBEnd of changes for service refresh 2
    The adapters must be interconnected by a suitable high-performance switch.
  • You must install OpenFabrics Enterprise Distribution (OFED) v1.5.3 or later, which can be downloaded from the OpenFabrics Alliance website: https://www.openfabrics.org/index.php/resources/ofed-for-linux-ofed-for-windows/ofed-overview.html. During run time, symbols from the following libraries are dynamically located and loaded: librdmacm.so and libibverbs.so. These libraries are provided by the OFED run time.
  • Users must have adequate permissions to lock memory on both host systems. RDMA socket buffers must be locked, or pinned, to avoid the operating system swapping out the memory pages during data transfers.

About this task

JSOR is enabled on the client and server system by specifying a system property on the command line. The system properties are different for plain stream socket applications and NIO channel applications. For detailed information about the construction of the configuration file, config_file, see -Dcom.ibm.net.rdma.conf (Linux only) or -Dcom.ibm.nio.rdma.conf (Linux only). Start of changes for service refresh 2 fix pack 10Support for NIO channel applications is added in this update.End of changes for service refresh 2 fix pack 10 To enable JSOR, complete the following steps:

Procedure

  1. Create the server configuration file on the server system with an accept rule.
  2. Create the client configuration file on the client system with a connect rule.
  3. Start the server, specifying the command-line system property -Dcom.ibm.net.rdma.conf=server_configuration_file or -Dcom.ibm.nio.rdma.conf=server_configuration_file
  4. Start the client, specifying the command-line system property -Dcom.ibm.net.rdma.conf=client_configuration_file or -Dcom.ibm.nio.rdma.conf=client_configuration_file
  5. Optional: If you want to override the bind address that is used by the system, you must use the -Dcom.ibm.net.rdma.preferredAddress or -Dcom.ibm.nio.rdma.preferredAddress command-line system property, with a valid InfiniBand address. For more information, see -Dcom.ibm.net.rdma.preferredAddress (Linux only) or -Dcom.ibm.nio.rdma.preferredAddress (Linux only).
  6. Optional: If you want your server system to accept connections from TCP clients when RDMA connections are not possible, specify the system property -Dcom.ibm.net.rdma.tcpFallback=true. For more information about this property, see -Dcom.ibm.net.rdma.tcpFallback (Linux only).
    1. Optional: You can enforce a time that an RDMA server must wait for RDMA services before it switches to TCP communications. For more information, see -Dcom.ibm.net.rdma.tcpFallbackWaitTime (Linux only).
    This option is not supported for NIO channel applications.

Results

The plain or NIO stream sockets connection between the Java application server and client transparently switches to RDMA, providing the rules match at both ends of the connection.
  • Server: The switch to an RDMA connection occurs during the bind / accept stage.
  • Client: The switch to an RDMA connection occurs during the bind / connect stage.
When connected, the server and client endpoints communicate and transfer data by using the R-Sockets protocol.

If you experience problems when you are trying to establish a connection between RDMA-enabled systems, you can use the JSOR trace facility to determine where the process is failing. For more information, see JSOR problem determination (Linux only).

Example

Create file server.conf on the Java application server, host1, with the following configuration rule:
rdma accept host1 1500 host2
All the connections from host2 accepted by host1 on the port 1500 are RDMA connections.
Create file client.conf on the Java application client, host2, with the following configuration rule:
rdma connect host1 1500
All the connections that are made to host1 on port 1500 are RDMA connections.
Start the Java application server in RDMA mode, by specifying the following on the command line: 
java -Dcom.ibm.net.rdma.conf=<path>/server.conf Server
Or:
java -Dcom.ibm.nio.rdma.conf=<path>/server.conf NIOServer
Where <path> is the fully qualified path to the configuration file.
Start the Java client application in RDMA mode, by specifying the following on the command line: 
java -Dcom.ibm.net.rdma.conf=<path>/client.conf Client
Or:
java -Dcom.ibm.nio.rdma.conf=<path>/client.conf NIOClient
Where <path> is the fully qualified path to the configuration file.

What to do next

To optimize the performance of your RDMA-enabled applications, you should set and size resources for connection and data transfer. For managing socket send and receive buffers, and queue sizes, JSOR includes a set of Java environment variables. For more information, see JSOR environment settings (Linux only).