Using InfiniBand ports with IPoIB

Complete these procedures to set up IPoIB.

1. Create the ifcfg-ibX script with suitable entries to use InfiniBand ports with IPoIB. For example:

   DEVICE=ib0 
   BOOTPROTO=static 
   IPADDR=192.168.0.17 
   NETMASK=255.255.254.0 
   NETWORK=192.168.0.0 
   BROADCAST=192.168.0.255 
   ONBOOT=yes 

   Note: With later Red Hat Enterprise Linux 6.x releases, use PREFIX instead of NETMASK. For example, instead of NETMASK=255.255.255.0, use PREFIX=24.
2. Activate the script with the following RDMA script:

   openibd/rdma restart

3. Confirm that the IPoIB interfaces are available by running the following command and verifying the output:

   ip addr show ib0 
   
   24: ib0:  <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 2044 qdisc pfifo_fast state UP qlen 256 
   link/infiniband 80:32:00:48:fe:80:00:00:00:00:00:00:00:02:c9:03:00:01:28:ed brd 00:ff:ff:ff:ff:12:40:1b:ff:ff:00:00:00:00:00:00:ff:ff:ff:ff 
   inet 192.168.0.198/24 scope global ib0 
   inet6 fe80::202:c903:1:28ed/64 scope link 
   valid_lft forever preferred_lft forever 

4. Verify that the nodes can communicate using IPoIB with the following command:

   ping -c 3 10.10.10.2 
   
   PING 10.10.10.2 (10.10.10.2) 56(84) bytes of data. 
   64 bytes from 10.10.10.2: icmp_seq=1 ttl=64 time=0.110 ms 
   64 bytes from 10.10.10.2: icmp_seq=2 ttl=64 time=0.057 ms 
   64 bytes from 10.10.10.2: icmp_seq=3 ttl=64 time=0.056 ms 
   . --- 10.10.10.2 ping statistics --- 
   3 packets transmitted, 3 received, 0% packet loss, time 1998ms 
   rtt min/avg/max/mdev = 0.056/0.074/0.110/0.026 ms