Interrupt coalescing

To avoid flooding the host system with too many interrupts, packets are collected and one single interrupt is generated for multiple packets. This is called interrupt coalescing.

For receive operations, interrupts typically inform the host CPU that packets have arrived on the device's input queue. Without some form of interrupt moderation logic on the adapter, this might lead to an interrupt for each incoming packet. However, as the incoming packet rate increases, the device driver finishes processing one packet and checks to see if any more packets are on the receive queue before exiting the driver and clearing the interrupt. The driver then finds that there are more packets to handle and ends up handling multiple packets per interrupt as the packet rate increases, which means that the system gets more efficient as the load increases.

However, some adapters provide additional features that can provide even more control on when receive interrupts are generated. This is often called interrupt coalescing or interrupt moderation logic, which allows several packets to be received and to generate one interrupt for several packets. A timer starts when the first packet arrives, and then the interrupt is delayed for n microseconds or until m packets arrive. The methods vary by adapter and by which of the features the device driver allows the user to control.

Under light loads, interrupt coalescing adds latency to the packet arrival time. The packet is in host memory, but the host is not aware of the packet until some time later. However, under higher packet loads, the system performs more efficiently by using fewer CPU cycles because fewer interrupts are generated and the host processes several packets per interrupt.

For AIX® adapters that include the interrupt moderation feature, you should set the values to a moderate level to reduce the interrupt overhead without adding large amounts of latency. For applications that might need minimum latency, you should disable or change the options to allow more interrupts per second for lower latency.

The Gigabit Ethernet adapters offer the interrupt moderation features. The FC 2969 and FC 2975 GigE PCI adapters provide a delay value and a buffer count method. The adapter starts a timer when the first packet arrives and then an interrupt occurs either when the timer expires or when n buffers in the host have been used.

The FC 5700, FC 5701, FC 5706, and FC 5707 GigE PCI-X adapters use the interrupt throttle rate method, which generates interrupts at a specified frequency that allows for the bunching of packets based on time. The default interrupt rate is 10 000 interrupts per second. For lower interrupt overhead, you can set the interrupt rate to a minimum of 2 000 interrupts per second. For workloads that call for lower latency and faster response time, you can set the interrupt rate to a maximum of 20 000 interrupts. Setting the interrupt rate to 0 disables the interrupt throttle completely.

The 10 Gigabit Ethernet PCI-X adapters (FC 5718 and 5719) have an interrupt coalescing option (rx_int_delay) with a delay unit of 0.82 microseconds. The actual length of the delay is determined by multiplying 0.82 by the value set in rx_int_delay. This option is disabled by default (rx_int_delay=0) because testing concluded that at the higher input rate of these adapters, coalescing interrupts do not help performance.