RDMA connectivity in Db2 pureScale environments

Similar to TCP/IP, an RDMA connection request requires at least a target IP address and a port value. The target IP address is the target network address of the connection request that is sent. The port value is for specifying which service (the server application) within the target network location that the connection request is intended for. Therefore, the port number must be unique for each network interface. For Db2 pureScale® environments, if there are multiple Host Channel Adapters (HCAs) defined for the CF, the same port value is used for all the interfaces. The port value is the RDMA connection qualifier that associates a connection request to a specific server application.

The target IP address is specified using either a name format or an IPv4 format. If a name format is used, the name to IP address resolution must be available by using a local /etc/hosts file or by using a DNS server. Typically in a Db2 pureScale installation, a local /etc/hosts file is used to provide the mapping between network names and IP addresses.

Note: The IPv6 address format is not supported.

When the CF is started, the port values are automatically retrieved from the /etc/services file at the CF host. The CF port entries in the /etc/services file use the following convention:

DB2CF_<instanceName>_MGMT     <mgmnt_port>/tcp
DB2CF_<instanceName>          <port>/tcp

The <instance_name> is the name of the Db2 pureScale instance. The <port> is the port value that is used for the RDMA connection. The <mgmnt_port> is the port value that is used for the TCP/IP management port connectivity.

Note: During instance creation, unused port values are detected and required entries are created in the /etc/services file to reserve the port values for the Db2 instance.

After you issue the db2start command, the Db2 member attempts to make a number of RDMA connections to the CF. The member determines the target CF address (the IP address) by using the network names (netnames) associated with the CF entry in the db2nodes.cfg file. If there are multiple HCAs used with the CF, there are multiple netnames for the same CF entry in the db2nodes.cfg file.

From the member host, the CF server port value to use on the connect request is retrieved from the local /etc/services file. This value must match the one used to start CF for the connection to work successfully. It is important to ensure that the /etc/services file content is identical for all the hosts in the Db2 pureScale cluster. Checking the file content prevents a situation where the member and the CF arrive at a different port value after the port name resolution.

The uDAPL configuration information is maintained in the /etc/dat.conf file on each host. It is important to verify that the uDAPL interface is configured correctly on each host. On RHEL, the dat.conf file is located under /etc/ofed for RHEL 5.x and /etc/rdma for RHEL 6.x. The dat.conf file is not required for RHEL 8.x and higher.