Internet Protocol over InfiniBand (IPoIB)
Internet protocol (IP) packets can be sent over an InfiniBand (IB) interface. This transport is accomplished by encapsulating IP packets of IB packets using a network interface.
In order to use IP over IB, you must install and configure the InfiniBand connection manager (ICM) driver and at least one IB device in the system. To see if an IB device is already installed, run the lsdev -C | grep iba command. The name of the fileset containing the IB interface is: devices.common.IBM.ib. The devices.chrp.IBM.lhca fileset is an example of a currently supported adapter fileset.
To configure an ICM driver, refer to Configuring an InfiniBand Communication Manager driver.
In order to create the InfiniBand interface (IB IF), the IB IF must be able to join an existing broadcast-multicast group with a user-provided PKEY (or a default PKEY = 0xFFFF is used if the user did not provide one) and a user provided Q_Key (or a default Q_Key = 0x1E is used if the user did not provide one). A broadcast-multicast group is a multicast group that the interface must join in order to send broadcast and ARP packets. If such a broadcast-multicast group does not exist, or can not be created by the interface, the creation of the IB IF will fail.
- interface name
- adapter name
- port number
- interface IP address
- internet address
- network mask
- MTU size (equal to the MTU desired, less 4 bytes for the IB header)
- state
- Send and Receive queue size (default is 4000)
- Multicast Queue Key
- Super packet on or off
$ /usr/sbin/mkiba -i ib0 -p 1 -A iba0 -a 1.2.3.8 [-P -1 -S "up" -m "255.255.254.0" -M 2044]
where:
Item | Description |
---|---|
-M 2044 |
Maximum transmit unit. |
-m "255.255.254.0" |
Net mask. |
-p 1 |
Port number (defaults to 1 if not provided). |
-A iba0 |
IB device name. |
-a 1.2.3.8 |
IF IP address. |
-i ib0 |
Interface name. |
-P -1 |
Partition key (defaults to PKEY if not provided. After the interface is created, PKEY cannot be changed; the user must obtain a non-default PKEY from the network administrator.) |
-S "up" |
Interface state. |
-q 8000 |
Receive and Transmit Queues size (each). |
-Q 0x1E |
Multicast Queue Key assigned to the Multicast group (defaults to Q_KEY = 0x1E if not provided). |
-k "on" |
Superpacket will allow the interface TCP/IP MTU to be 64K. It has to be enabled in the remote host also to be able to work. |
$ smitty inet
After the Network Interface Selection menu is displayed, follow this procedure:
- Select Add a Network Interface or Change / Show Characteristics of a Network Interface. The Add a Network Interface menu displays.
- In the Add a Network Interface menu, select Add an IB Network Interface. The Add an IB Network Interface menu displays.
- In the Add an IB Network Interface menu, make the necessary changes and press Enter.