Overview of a problem report

For Power9 hardware failure can be an IBM Storage Scale System 5000 I/O node or an management server node or a protocol node. Hardware failure event can be DIMM failure, power supply failure, or fan failure in Power9 nodes. Any of the failure events reported by the Power nodes are recorded in OPAL log. IBM Storage Scale System hardware call home reads the OPAL events, which need service and it is a call home event.

IBM Storage Scale System 5000 and Power management server hardware failure event raised in OPAL log reported by event type

  • disk_call_home, disk_call_home2 – GNR disk needs replacement.
  • nodeEvent - Any Power9 node hardware failure which is reported in the OPAL log is a call home event that is reported with this event.
  • raid_root_disk_bad – Power boot disk failure.
  • raid_sas_adapter_bad, sas_adapter_down_trained, sas_adapter_failed, sas_cable_failed - Power SAS adapter failures.
  • sas_adapter_missing – Expected SAS adapter is not dected by the Power system.
  • power_supply_failed, power_supply_fan_critical, power_supply_iout_critical, power_supply_vout_critical, ps_fan_failed – Expansion enclosure PSU failures.
  • temp_bus_failed, temp_sensor_failed, voltage_bus_failed, voltage_sensor_failed, temp_bus_failed, temp_sensor_failed – Expansion enclosure sensor failures.
  • power_high_current, power_high_voltage – Expansion enclosure sensor reporting input power outside expected range.

Similarly, IBM Storage Scale System 3000, IBM Storage Scale System 3200, IBM Storage Scale System 3500, IBM® Storage Scale System Utility Node, IBM Storage Scale System 6000 x86 nodes can also report any of the hardware failure events with mmhealth.

IBM Storage Scale System 3000 supported hardware events

  • disk_call_home, disk_call_home2 – GNR disk needs replacement.
  • bootDrvFail - When boot drive failed at canister.
  • canFailed - When canister failed.
  • bootDrvMissing - When boot drive missing.
  • bootDrvSmtFailed - When boot drive smart failed.
  • canFanFailed - When canister FAN stopped working.
  • fanFailed - When FAN failed.
  • psFailed - Power supply failure.
  • psFanFailed - Power supply FAN failure.

IBM Storage Scale System 3200, IBM Storage Scale System 3500, IBM Storage Scale System Utility Node, and IBM Storage Scale System 6000supported hardware events

  • disk_call_home, disk_call_home2 – GNR disk needs replacement.
  • nvmeof_raw_disk_smart_failed, nvmeof_raw_disk_smart_unknown – non GNR disk needs replacement.
  • bootdrive_mirror_degraded, bootdrive_mirror_failed – Boot drive mirror no longer redundant.
  • bootdrive_missing – Expected boot drive not detected by the system.
  • bootdrive_power_mode_degraded – Boot drive power mode is not at the expected value.
  • bootdrive_smart_failed – Pro active failure reported by the boot drive.
  • can_fan_missing – Utility Node internal fan not detected by the system.
  • canister_failed – Node hardware failure.
  • pair_canister_failed – Partner node is not reachable.
  • can_power_supply_failed, can_power_supply_missing - Utility Node PSU failures.
  • can_temp_bus_failed, can_temp_sensor_failed, temp_bus_failed, temp_sensor_failed, voltage_bus_failed, voltage_sensor_failed – Enclosure sensor failures.
  • coin_battery_low, coin_battery_missing – CMOS battery failures.
  • cpu_unit_in_wrong_slot, cpu_unit_missing, cpu_unit_speed_wrong – CPU health check errors.
  • dimm_config_mismatch, dimm_module_in_wrong_slot, dimm_module_missing, dimm_module_size_wrong, dimm_module_speed_wrong – DIMM health check errors.
  • enc_temp_critical – Enclosure reporting elevated temperature, thermal shutdown might follow.
  • esm_failed, expander_failed – Expansion IO module failures.
  • fan_absent – Fan not detected in fan bay.
  • can_fan_failed – Utility node internal fan failure.
  • fan_failed, fan_fault_indicated, fan_speed_high, fan_speed_low – enclosure fan failures.
  • power_high_current, power_high_voltage – Enclosure sensor reporting input power outside expected range.
  • power_supply_config_mismatch – Incompatible of PSU types in the same enclosure.
  • power_supply_failed, power_supply_fan_critical, power_supply_iout_critical, power_supply_vout_critical, ps_fan_failed – PSU failures.
  • sas_adapter_down_trained, sas_adapter_failed, sas_adapter_missing, sas_cable_failed – Connectivity to expansion enclosure failures.