IBM Support

POWER9 POWERVM Incomplete State During I/O Concurrent Repair or Dynamic LPAR of IBM i owned hardware

Troubleshooting


Problem

ATTENTION
 
Do not use concurrent maintenance, or dynamic LPAR on POWER9 servers under the following conditions:
  1. The resource physically owned by an IBM i partition.
    • If no IBM i partitions are present, this document does not apply.
    • If the resources are owned by VIOS, AIX, or Linux®, this document does not apply.
    • If the resources are Single Root I/O Virtualization (SR-IOV) resources, this document does not apply.
  2. The resource is a physical PCIe adapter, an EMX0 cable card, active optical cable (AOC), fanout module, chassis management card, or midplane.
  3. The firmware fix and IBM i PTFs listed in this document are not applied.
Without the fix applied, the server can go to an incomplete state during the operation requiring server level IPL to recover.

An incomplete state can occur only on a POWER9 PowerVM managed server at some time after one of the following operations:
 
  1. Dynamic Logical Partition (DLPAR) operation to remove or move a physical PCIe adapter from a logical partition (LPAR).
  2. Concurrent replacement of a physical PCIe adapter assigned to an LPAR.
  3. Concurrent replacement of a Cable Card, cable, fanout module, mid-plane, or chassis management card from an EMX0 I/O drawer.
The condition is set up when one of these operations is blocked by the hypervisor because the resource is still in use.  When concurrent maintenance is performed, the hypervisor ends with a return code of 0x0300.  When the partition is powered down and the operation retried, the server can go to an incomplete state.  Currently, the only known trigger for this problem resides in IBM i operating system.
 
Once the server is in an incomplete state from this defect, the server must be re-IPLed to recover management full operational capability.

Resolving The Problem

A firmware fix can be applied to prevent this incomplete state from occurring.  This fix is provided in the following system firmware fix levels:
FW Level Released or Planned Date
FW910.50 Vx910_151 Released 8 April 2020
FW920.50 Vx920_118 Released 25 November 2019
FW930.20 Vx930_101 Released 27 February 2020
FW940.00 Vx940_027 Released 22 November 2019
Future firmware releases contain this fix.
 
The following IBM i APARs must be applied to partitions owning PCIe hardware in order to prevent the problem from being triggered.
 
APAR MA47837 LIC-OTHER-SRCB6005120-INCORROUT POWER9 DLPAR-add fails
  R740 MF66968 UP20/02/24
  R730 MF66695 UP20/02/10
  R720 MF66544 UP20/02/10

APAR MA47943 LIC-OTHER-SRCB6006965-INCORROUT DLPAR remove after add
  R740 MF66995 UP20/02/24
  R730 MF66865 UP19/11/27
  R720 MF67112 UP20/02/28

Document Location

Worldwide

[{"Type":"MASTER","Line of Business":{"code":"LOB57","label":"Power"},"Business Unit":{"code":"BU058","label":"IBM Infrastructure w\/TPS"},"Product":{"code":"SWG60","label":"IBM i"},"Platform":[{"code":"PF012","label":"IBM i"}],"Version":"7.1.0"},{"Business Unit":{"code":"BU058","label":"IBM Infrastructure w\/TPS"},"Product":{"code":"TI0007I","label":"Power System E980 Server (9080-M9S)"},"Component":"","Platform":[{"code":"PF012","label":"IBM i"}],"Version":"All Versions","Edition":"","Line of Business":{"code":"LOB57","label":"Power"}},{"Business Unit":{"code":"BU058","label":"IBM Infrastructure w\/TPS"},"Product":{"code":"TI0007E","label":"Power System E950 Server (9040-MR9)"},"Component":"","Platform":[{"code":"PF012","label":"IBM i"}],"Version":"All Versions","Edition":"","Line of Business":{"code":"LOB57","label":"Power"}},{"Business Unit":{"code":"BU058","label":"IBM Infrastructure w\/TPS"},"Product":{"code":"TI0005E","label":"Power System S914 Server"},"Component":"","Platform":[{"code":"PF012","label":"IBM i"}],"Version":"All Versions","Edition":"","Line of Business":{"code":"LOB57","label":"Power"}},{"Business Unit":{"code":"BU058","label":"IBM Infrastructure w\/TPS"},"Product":{"code":"TI0005G","label":"Power System S922 Server (9009-22A)"},"Component":"","Platform":[{"code":"PF012","label":"IBM i"}],"Version":"All Versions","Edition":"","Line of Business":{"code":"LOB57","label":"Power"}},{"Business Unit":{"code":"BU058","label":"IBM Infrastructure w\/TPS"},"Product":{"code":"HW1A1","label":"IBM Power Systems"},"Component":"","Platform":[{"code":"PF012","label":"IBM i"}],"Version":"All Versions","Edition":"","Line of Business":{"code":"LOB57","label":"Power"}}]

Document Information

Modified date:
07 December 2021

UID

ibm12404215