Configuring HiperSockets in Linux LPARs
HiperSockets™ are currently only supported at the LPAR level. To configure an LPAR with a HiperSockets interface the following steps must be completed:
- Select which HiperSockets configuration(s) to enable.
- Modify the kernel command line in the boot configuration file to enable one or more HiperSockets configurations.
- Apply/commit the changes for the next/future restarts.
- Restart the system to activate the enabled HiperSockets configuration(s).
- Create a network configuration file for each newly enabled HiperSockets device.
HiperSockets are supported in four different configurations. The functional difference in the configurations is the MTU size that a specific HiperSockets configuration supports. IBM® Z currently supports HiperSockets with MTU sizes of 8, 16, 32 and 56 Kbytes.
The following table contains specific system attributes associated with each MTU size. Choose the Device Address field that corresponds to the HiperSockets MTU size to be enabled on the kernel command line.
| Card type | CHPID | Device Address |
|---|---|---|
| FSZ 16K MTU 8K | FB | 8000 - 8002 |
| FSZ 24K MTU 16K | FC | 8100 - 8102 |
| FSZ 40K MTU 32K | FD | 8200 - 8202 |
| FSZ 64K MTU 56K | FE | 8300 - 8302 |
One of the values typically specified to the kernel boot parameter key is the cio_ignore= keyword. cio_ignore is used to provide a blacklist to the kernel of visible devices that should be ignored when the kernel boots. Some SAN configurations present all available devices to any system attached to the SAN. cio_ignore provides an easy way to specify which devices should or should not be ignored for each boot configuration. Normally the cio_ignore= key is followed by the all value. This instructs the kernel to ignore all devices. The all value is then typically followed by one or more device addresses (separated by a comma) that are not to be ignored. Devices not to be ignored begin with a not symbol (!) symbol directly in front of the address. For example, !0.0.0001 implies that this device should not be ignored or said another way, and it adds the device to the kernel's whitelist.
Network devices are comprised of three subchannel devices (read, write, and data). Each subchannel device has a unique address. All three subchannel addresses must be specified in read, write, data order.
To add a HiperSockets device with a 56K MTU size to the kernel's whitelist, the device addresses for that configuration must be added using the cio_ignore= key. For example, suppose the system's cio_ignore= key looks like this:
… cio_ignore=all,!0.0.0001 …
The kernel device whitelist currently only allows device 0.0.0001 to be visible at kernel boot. To make the HiperSockets device with MTU 56k visible, the cio_ignore= key would need to be changed to this:
… cio_ignore=all,!0.0.0001,!0.0.8300,!0.0.8301,!0.0.8302 …
See Modifying the kernel command line parameters to apply this change and make it active.
In these LPAR configurations, the same HiperSockets configuration (MTU 56k) was applied to both LPARs. After rebooting, each LPAR should have the configured HiperSockets device available.
A network configuration file was then created for the new HiperSockets device in each LPAR. The file /etc/sysconfig/network-scripts/ifcfg_hs56k_0 was created on both LPARs with the following contents:
DEVICE=hs56k_0 NM_CONTROLLED="no" ONBOOT=yes BOOTPROTO=static NETMASK=255.0.0.0 IPADDR=172.31.56.x SUBCHANNELS=0.0.8300,0.0.8301,0.0.8302 NETTYPE=qeth TYPE=Ethernet OPTIONS="checksumming=sw_checksumming buffer_count=128 layer2=0" MACADDR='02:bb:83:xx:00:04'
The device name is hs56k_0 to represent the first 56K HiperSockets interface. Other 56K HiperSockets devices could be created using the next group of three subchannel device addresses (in this example, 8303-8305) and so on.
For convenience, the IP addresses were made static and used 172.31.x.x part of the private 20-bit block Class B address range (172.16.0.0 - 172.31.255.255).1 The third octet in the IP address was set to the MTU size (in KBs) of the HiperSockets device, and the last octet used a unique number for the HiperSockets in each LPAR. The HiperSockets in the KVM host was set to IP address 172.31.56.1, and the HiperSockets in the other LPAR used 172.31.56.2. In order for the HiperSockets to communicate, the devices must be on the same subnet. The subnet used here is a Class B. The IP addresses and subnet class was arbitrarily chosen to meet the needs of this configuration. HiperSockets can be configured using any suitable IP addresses and subnets.
In a similar fashion the MACADDR= for each HiperSockets device was set with the fourth byte displaying the LPAR number.
The OPTIONS= keyword specifies the configured OSI model. HiperSockets can support Layer 2 or Layer 3. The option Layer2=0 really means that Layer 3 is active. Layer 3 is used to emulate what a z/OS® LPAR would be required to use.
IBM Z supports different types of network devices. For a HiperSocket device, specify the settings NETTYPE=qeth and TYPE=ethernet.
Once these files have been created and modified accordingly for the device name, IP, subnet, and MacAddr desired, save and exit the files.
On each LPAR, run the following command to bring the HiperSockets interface online:
[root@kvmhost ~]# ifup hs56k_0
Once the HiperSockets interface in each LPAR is online, each LPAR should be able to ping the other:
[root@kvmhost ~]# ping 172.31.56.2 PING 172.31.56.2 (172.31.56.2) 56(84) bytes of data. 64 bytes from 172.31.56.2: icmp_seq=1 ttl=64 time=0.055 ms 64 bytes from 172.31.56.2: icmp_seq=2 ttl=64 time=0.029 ms [root@otherlpar ~]# ping 172.31.56.1 PING 172.31.56.1 (172.31.56.1) 56(84) bytes of data. 64 bytes from 172.31.56.1: icmp_seq=1 ttl=64 time=0.042 ms 64 bytes from 172.31.56.1: icmp_seq=2 ttl=64 time=0.028 ms
At this point the HiperSockets interfaces should be fully functional and ready to be used by the KVM guest to communicate with other LPARs.