Network setup

Edit online

A single 10G network link using modified settings was used for communication to the storage servers. Each LPAR utilized a different physical OSA network adapter card.

The network settings were modified using information obtained from KVM Network Performance - Best Practices and Tuning Recommendations (https://www.ibm.com/docs/en/linux-on-systems?topic=kvm-network-performance-best-practices-tuning-recommendations).

The tuning was applied through two files. One file adjusted the sysctl values as shown in Figure 1. The second file added a udev rule file to adjust the transmit queue and MTU values Figure 2.

The sysctl.conf file Figure 1 configured with an rmem/wmem maximum value of 16M and a fin_timeout of 10 seconds. In the measurement environment the default congestion control is cubic, so it was not set through the systctl file.

Figure 1. Example sysctl configuration file

$ cat /etc/sysctl.d/90-sysctl-netbp-example.conf
net.core.netdev_max_backlog = 25000

net.ipv4.tcp_fin_timeout = 1
net.ipv4.tcp_max_tw_buckets = 450000

net.core.rmem_max = 16777216
net.core.wmem_max = 16777216
net.ipv4.tcp_rmem = 4096 87380 16777216
net.ipv4.tcp_wmem = 4096 16384 16777216
net.ipv4.tcp_tw_reuse = 1

net.ipv4.tcp_limit_output_bytes = 131072
net.ipv4.tcp_low_latency = 0

net.ipv4.ip_forward=1

Figure 2. Example udev rules file

$ cat /etc/udev/rules.d/etc/udev/rules.d/90-netbp-settings-examples.rules
#
# Replace "10G_NET_DEV" with actual network device name.
# Set MTU and txqueuelen
#

KERNEL=="<10G_NET_DEV>", RUN+="/sbin/ip link set %k txqueuelen 2500"
KERNEL=="<10G_NET_DEV>", RUN+="/sbin/ip link set %k mtu 8192"