Broadcom netxtreme II ethernet nics - IBM System Cluster 1350 (Type 1410)

Resolving The Problem

Source

RETAIN tip: H134177

Symptom

When running heavy traffic on the second on-board interface on systems using Broadcom NetXtreme II Ethernet Network Interface Controllers (NIC), packet drops can occur and rx_fw_discards can be seen in the ethtool -S statistics under RHEL 5.3 x86_64 and using the bnx2 1.8.2b driver.

Affected configurations

The system may be any of the following IBM Servers:

• IBM System Cluster 1350, type 1410, any model

The system is configured with at least one of the following:

• Red Hat Enterprise Linux 5, Update 3

This tip is not option specific.

Note: This does not imply that the network operating system will work under all combinations of hardware and software.

Workaround

If this behavior occurs, increase the ring buffer by typing this command:

The default with RHEL 5.3 x86_64 and the bnx2 1.8.2b driver is 255 and the max value is 1020.

Additional information

Under a very heavy load of packets received, the NIC receive buffer can overflow before the OS can read the packets off the NIC, which results in the rx_fw_discards occurring.

Increasing the ring buffer size from the default can mitigate this problem. Increasing the size of the ring buffer will reduce the amount of free memory on the system.