ARP cache tuning

Edit online

The Address Resolution Protocol (ARP) is used to map 32-bit IPv4 addresses into a 48-bit host adapter address required by the data link protocol.

ARP is handled transparently by the system. However, the system maintains an ARP cache, which is a table that holds the associated 32-bit IP addresses and its 48-bit host address. You might need to change the size of the ARP cache in environments where large numbers of machines (clients) are connected. This can be done using the no and netstat commands.

The no command configures network tuning parameters and its ARP-related tunable parameters are:

  • arpqsize = 12
  • arpt_killc = 20
  • arptab_bsiz = 7
  • arptab_nb = 149

The ARP table size is composed of a number of buckets, defined by the arptab_nb parameter. Each bucket holds the number of entries defined in the arptab_bsiz parameter. The defaults are 149 buckets with 7 entries each, so the table can hold 1043 (149 x 7) host addresses. This default setting will work for systems that would be communicating with up to 1043 other machines concurrently on the IP network. If a server connects to more than 1043 machines on the network concurrently, then the ARP table will be too small, causing the ARP table to thrash and resulting in poor performance. The operating system then must purge an entry in the cache and replace it with a new address. This requires the TCP or UDP packets to be queued while the ARP protocol exchanges this information. The arpqsize parameter determines how many of these waiting packets can be queued by the ARP layer until an ARP response is received back from an ARP request. If the ARP queue is overrun, outgoing TCP or UDP packets are dropped.

ARP cache thrashing might have a negative impact on performance for the following reasons:

  1. The current outgoing packet has to wait for the ARP protocol lookup over the network.
  2. Another ARP entry must be removed from the ARP cache. If all of the addresses are needed, another address is required when the host address that is deleted has packets sent to it.
  3. The ARP output queue might be overrun, which could cause dropped packets.

The arpqsize, arptab_bsiz, and arptab_nb parameters are all reboot parameters, meaning that the system must be rebooted if their values change because they alter tables that are built at boot time or TCP/IP load time.

The arpt_killc parameter is the time, in minutes, before an ARP entry is deleted. The default value of the arpt_killc parameter is 20 minutes. ARP entries are deleted from the table every number of minutes defined in the arpt_killc parameter to cover the case where a host system might change its 48-bit address, which can occur when its network adapter is replaced. This ensures that any stale entries in the cache are deleted, as these would prevent communication with such a host until its old address is removed. Increasing this time would reduce ARP lookups by the system, but can result in holding stale host addresses longer. The arpt_killc parameter is a dynamic parameter, so it can be changed without rebooting the system.

The netstat -p arp command displays the ARP statistics. These statistics show how many total ARP requests have been sent and how many packets have been purged from the table when an entry is deleted to make room for a new entry. If this purged count is high, then your ARP table size should be increased. The following is an example of the netstat -p arp command. 
# netstat -p arp

arp:  
     6 packets sent
     0 packets purged
You can display the ARP table with the arp -a command. The command output shows those addresses that are in the ARP table and how those addresses are hashed and to what buckets. 
 ? (10.3.6.1) at 0:6:29:dc:28:71 [ethernet] stored 
                                                   
bucket:    0     contains:    0 entries            
bucket:    1     contains:    0 entries            
bucket:    2     contains:    0 entries            
bucket:    3     contains:    0 entries            
bucket:    4     contains:    0 entries            
bucket:    5     contains:    0 entries            
bucket:    6     contains:    0 entries            
bucket:    7     contains:    0 entries            
bucket:    8     contains:    0 entries            
bucket:    9     contains:    0 entries            
bucket:   10     contains:    0 entries            
bucket:   11     contains:    0 entries            
bucket:   12     contains:    0 entries            
bucket:   13     contains:    0 entries            
bucket:   14     contains:    1 entries            
bucket:   15     contains:    0 entries            


...lines omitted...

There are 1 entries in the arp table.