IBM Support

Issue identified with FW940.40 and SR-IOV Shared mode FC EC2R / EC2S, EC2T / EC2U, EC3L / EC3M, EC66 / EC67

Release Notes


Abstract

For Power 9 systems with IO adapters with Feature Codes and CCINs #EC2R/#EC2S with CCIN 58FA; #EC2T/#EC2U with CCIN 58FB; #EC3L/#EC3M with CCIN 2CEC; and #EC66/#EC67 with CCIN 2CF3, running in SR-IOV Shared mode, server firmware level FW940.40 should not be applied.

Content

An issue has been found, involving FW940.40 (Vx940_087_027) and certain SR-IOV Shared mode adapters.  The adapter types affected have Feature Codes and CCINs   #EC2R/#EC2S with CCIN 58FA; #EC2T/#EC2U with CCIN 58FB; #EC3L/#EC3M with CCIN 2CEC; and #EC66/#EC67 with CCIN 2CF3, running in SR-IOV Shared mode configuration.  The issue can lead to network instability over these adapters.  Symptoms include multicast and broadcast packets not properly routing out the physical port, potentially causing a ping failure or an inability to establish TCP connections.
The problem can occur at initialization of the SR-IOV Shared mode adapter.
If the application of the FW940.40 firmware is concurrent, there are extra instructions that describe the selective updating of the SRIOV adapter code.  The selective update triggers a restart of the adapter.  This restart might not initialize the data structures properly, leading to network instability.
If server firmware FW940.40 is disruptively applied (such as upgrading from FW930 or earlier), on the server restart, these SRIOV shared mode adapters might not initialize the data structures properly, leading to network instability.
Recommended Actions:
For POWER9 systems with Feature Codes and CCINs   #EC2R/#EC2S with CCIN 58FA; #EC2T/#EC2U with CCIN 58FB; #EC3L/#EC3M with CCIN 2CEC; and #EC66/#EC67 with CCIN 2CF3, running in SR-IOV Shared mode, server firmware level FW940.40 should not be applied.
For POWER9 systems that have FW940.40 installed, and that have these adapter types in SR-IOV Shared mode configuration, there are two options for dealing with the exposure:
  • The server firmware level could be removed.  If a disruptive removal is performed, no further action is needed.  If a concurrent removal is performed, a selective update of the SR-IOV shared mode adapters needs to be performed following the server firmware removal.
  • A restart of the adapter can be performed, if the adapter is showing network instability, to possibly relieve the issue. This can work since the data structure is properly initialized on some restarts of the adapter.
    Perform an SR-IOV disruptive dump to restart the adapter:  https://www.ibm.com/support/pages/node/667943
      use selector:  sriov {location code of the adapter} restart
      (this dump with restart disrupts any traffic on the adapter at the time of the restart)
This issue will be fully addressed in the next FW940 service pack.
Contact IBM Support if you have further questions.

[{"Type":"MASTER","Line of Business":{"code":"LOB08","label":"Cognitive Systems"},"Business Unit":{"code":"BU058","label":"IBM Infrastructure w\/TPS"},"Product":{"code":"HW1A1","label":"IBM Power Systems"},"ARM Category":[],"Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"All Versions"},{"Type":"MASTER","Line of Business":{"code":"LOB08","label":"Cognitive Systems"},"Business Unit":{"code":"BU058","label":"IBM Infrastructure w\/TPS"},"Product":{"code":"SGDMMD","label":"Power System AC922 Server (8335-GTC)"},"ARM Category":[],"ARM Case Number":[],"Platform":[{"code":"PF025","label":"Platform Independent"}]},{"Type":"MASTER","Line of Business":{"code":"LOB08","label":"Cognitive Systems"},"Business Unit":{"code":"BU058","label":"IBM Infrastructure w\/TPS"},"Product":{"code":"TI0005G","label":"Power System S922 Server (9009-22A)"},"ARM Category":[{"code":"a8m0z000000bpKLAAY","label":"Firmware"}],"Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"All Versions"},{"Type":"MASTER","Line of Business":{"code":"LOB57","label":"Power"},"Business Unit":{"code":"BU058","label":"IBM Infrastructure w\/TPS"},"Product":{"code":"TI0005E","label":"Power System S914 Server"},"ARM Category":[{"code":"a8m0z000000bpKLAAY","label":"Firmware"}],"Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"All Versions"},{"Type":"MASTER","Line of Business":{"code":"LOB08","label":"Cognitive Systems"},"Business Unit":{"code":"BU058","label":"IBM Infrastructure w\/TPS"},"Product":{"code":"HW1A1","label":"IBM Power Systems"},"ARM Category":[{"code":"a8m0z000000bpKLAAY","label":"Firmware"}],"Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"All Versions"},{"Type":"MASTER","Line of Business":{"code":"LOB08","label":"Cognitive Systems"},"Business Unit":{"code":"BU058","label":"IBM Infrastructure w\/TPS"},"Product":{"code":"HW1A1","label":"IBM Power Systems"},"ARM Category":[{"code":"a8m0z000000bpKLAAY","label":"Firmware"}],"Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"All Versions"},{"Type":"MASTER","Line of Business":{"code":"LOB08","label":"Cognitive Systems"},"Business Unit":{"code":"BU058","label":"IBM Infrastructure w\/TPS"},"Product":{"code":"TI0007E","label":"Power System E950 Server (9040-MR9)"},"ARM Category":[],"Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"All Versions"},{"Type":"MASTER","Line of Business":{"code":"LOB08","label":"Cognitive Systems"},"Business Unit":{"code":"BU058","label":"IBM Infrastructure w\/TPS"},"Product":{"code":"TI0007I","label":"Power System E980 Server (9080-M9S)"},"ARM Category":[],"Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"All Versions"},{"Type":"MASTER","Line of Business":{"code":"LOB08","label":"Cognitive Systems"},"Business Unit":{"code":"BU058","label":"IBM Infrastructure w\/TPS"},"Product":{"code":"TI0007I","label":"IBM Power System E980 (9080-M9S)"},"ARM Category":[],"ARM Case Number":[],"Platform":[{"code":"PF025","label":"Platform Independent"}]}]

Document Information

Modified date:
07 December 2021

UID

ibm16477612