TCP/IP tuning guidelines for NFS performance

Edit online

NFS uses UDP or TCP to perform its network I/O.

Ensure that you have applied the tuning techniques described in TCP and UDP performance tuning and Tuning mbuf pool performance. In particular, you should do the following:

  • Check for system error log entries by running the errpt command and looking for reports of network device or network media problems.
  • Ensure that the LAN adapter transmit and receive queues are set to the maximum values. See Tuning adapter resources for more information.
  • Check for Oerrs with the netstat -i command. A significant number of these errors might indicate that the transmit queue size for the related network device is not large enough.
  • Ensure that TCP and UDP socket buffer sizes are configured appropriately. The nfs_tcp_socketsize tunable of the nfso command controls the TCP socket buffer sizes, tcp_sendspace and tcp_recvspace, used by NFS. Similarly, the nfs_udp_socketsize tunable controls the UDP socket buffer sizes, udp_sendspace and udp_recvspace, used by NFS. Follow the guidelines described in TCP and UDP performance tuning for setting socket buffer size tunables. As with ordinary TCP and UDP tuning, the value of the sb_max tunable of the no command must be larger than the nfs_tcp_socketsize and nfs_udp_socketsize values. In general, you should find that the default values used in AIX should be adequate, but it does not hurt to check this. To check for UDP socket buffer overruns, run the netstat –s –p udp command and look for a large number of dropped packets being reported in the socket buffer overflows field.
  • Ensure that enough network memory is configured in the system. Run the netstat –m command and see if there are any requests for denied or delayed mbufs. If so, increase the number of mbufs available to the network. For more information on tuning a system to eliminate mbuf problems, see Tuning mbuf pool performance.
  • Check for general routing problems. Use the traceroute command to look for unexpected routing hops or delays.
  • If possible, increase the MTU size on the LAN. On a 16 Mb Gigabit Ethernet network for example, an increase in MTU size from the default 1500 bytes to 9000 bytes (jumbo frames) allows a complete 8 KB NFS read or write request to be transmitted without fragmentation. It also makes much more efficient use of mbuf space, reducing the probability of overruns.
  • Check for MTU size mismatches. Run the netstat -i command and check the MTU on the client and server. If they are different, try making them the same and see if the problem is eliminated. Also be aware that slow or wide area network equipment, like routers or bridges, between the machines might further fragment the packets to traverse the network segments. One possible solution is to try to determine the smallest MTU between source and destination, and change the rsize and wsize settings on the NFS mount to some number lower than the lowest-common-denominator MTU.
  • When running NFS Version 3 with TCP, and using the default of 32 KB or larger RPC sizes, you should set the nfs_rfc1323 option of the nfso command. This allows for TCP window sizes greater than 64 KB, and thus helps minimize waiting for TCP acknowledgments. The option must be set on each side of the TCP connection, for example on both the NFS server and client.
  • Check for very small inter-packet delays. There have been rare cases where this has caused problems. If there is a router or other hardware between the server and client, you can check the hardware documentation to see if the inter-packet delays can be configured. If so, try increasing the delay.
  • Check for large media speed mismatches. When packets are traversing two media with widely different speeds, the router might drop packets when taking them off the high speed network and trying to get them out on the slower network. This may occur, for instance, when a router is trying to take packets from a server on Gigabit Ethernet and send them to a client on 100 Mbps Ethernet. It may not be able to send out the packets fast enough on 100 Mbps Ethernet to keep up with the Gigabit Ethernet. Aside from replacing the router, one other possible solution is to try to slow down the rate of client requests and/or use smaller read/write sizes.
  • The maximum number of TCP connections allowed into the server can be controlled by the new nfs_max_connections option. The default of 0 indicates that there is no limit. The client will close TCP connections that have been idle for approximately 5 minutes, and the connection is reestablished when use warrants it. The server will close connections that have been idle for approximately 6 minutes.
  • The operating system provides an option to turn off the UDP checksum for NFS only. You can use the nfso command option, called udpchecksum. The default is 1, which means the checksum is enabled. You can achieve slight performance gains by turning it off, at the expense of increased chance of data corruption.