IT30305: DEVICES NOT WORKING AFTER PHYSICAL VSNAP UPGRADE TO 10.1.4 DUE TO MISSING FIRMWARE BINARIES

APAR status

• Closed as program error.

Error description

• Upgrading the vSnap to 10.1.4 will upgrade the Linux kernel to
  version 4.19.27-1.
  Any driver of any hardware device (network adapter, RAID
  adapter, ...) that relies on external firmware binaries may fail
  to load if the firmware package is out of date. The failure
  message during boot will typically indicate the filename of the
  missing firmware.
  
  The example below shows the messages seen when the driver for a
  QLogic 10Gb ethernet adapter is missing:
  [   5.342776] QLogic FastLinQ 4xxxx Core Module qed 8.33.0.20
  [   5.475198] qede_init: QLogic FastLinQ 4xxxx Ethernet Driver
  qede 8.33.0.20
  [   5.631316] qede 0000:06:00.0: Direct firmware load for
  qed/qed_init_values_zipped-8.37.2.0.bin failed with error -2
  [   5.646328] [qed_slowpath_start:1054()]Failed to find fw file
  - /lib/firmware/qed/qed_init_values_zipped-8.37.2.0.bin
  [   5.723491] qede: probe of 0000:06:00.0 failed with error -2
  [   6.415052] qede 0000:06:00.1: Direct firmware load for
  qed/qed_init_values_zipped-8.37.2.0.bin failed with error -2
  [   6.434673] [qed_slowpath_start:1054()]Failed to find fw file
  - /lib/firmware/qed/qed_init_values_zipped-8.37.2.0.bin
  [   6.454638] qede: probe of 0000:06:00.1 failed with error -2
  
  IBM Spectrum Protect Plus Versions Affected:
  IBM Spectrum Protect Plus 10.1.4
  
  Initial Impact: high
  
  Additional Keywords: SPP, SPPlus, TS002656778
  

Local fix

• 1. Boot vSnap from the older working kernel
  2. Run "yum update linux-firmware" to get the updated firmware
  or download and install the latest linux-firmware RPM available
  from RedHat download site.
  3. Rebuild the initramfs using "dracut -f -v"
  4. Reboot vSnap in the new kernel
  

Problem summary

• ****************************************************************
  * USERS AFFECTED:                                              *
  * IBM Spectrum Protect Plus level 10.1.4 and 10.1.5.           *
  ****************************************************************
  * PROBLEM DESCRIPTION:                                         *
  * See Error Description                                        *
  ****************************************************************
  * RECOMMENDATION:                                              *
  * Apply fixing level when available. This problem is currently *
  * projected to be fixed in IBM Spectrum Protect Plus level     *
  * 10.1.6. Note that this is subject to change at the           *
  * discretion of IBM.                                           *
  ****************************************************************
  

Problem conclusion

• When the upgraded Linux kernel v4.19 was installed alongside
  vSnap, the corresponding upgraded linux-firmware was not
  packaged. As a result, older firmware was included when building
  the boot image for the new kernel. During boot process, certain
  hardware which required newer firmware did not work. The problem
  has been resolved by packaging up-to-date linux-firmware package
  alongside the up-to-date kernel to ensure they stay in sync.
  

Temporary fix

Comments

APAR Information

• APAR is sysrouted FROM one or more of the following:

• APAR is sysrouted TO one or more of the following:

Fix information

• Fixed component name

  SP PLUS

• Fixed component ID

  5737SPLUS

Applicable component levels