IBM Support

Memory disabled with scrub failures in the system log - IBM Servers

Troubleshooting


Problem

When a failure is detected during early boot 'memory initialization' phase on a Dual In-Line Memory Module (DIMM), the Integrated Management Module (IMM) will log memory disabled as well as log scrub failure messages for that DIMM. Note: This event isnot a true Memory Scrub failure. This event is generated by the Unified Extensible Firmware Interface (UEFI) from a Power On Self Test (POST) Memory Failure, such as Failed Training or Failed MRC. This feature was added across all IMM systems in the Third Quarter 2011 life cycle release and will be incorporated in IMM2. For example, when the system detects the memory error on slot 8, the log will have scrub failure in the log as well. 10 I 01/01/1970; 01:00:59 0x806f040c2008ffff "Memory Device 8" disabled on subsystem "System Memory" 11 E 01/01/1970; 01:00:59 0x806f030c2008ffff Scrub Failure for "Memory Device 8" on subsystem "System Memory" Runtime memory failures that cannot be corrected will be logged with the following err

Resolving The Problem

Source

RETAIN tip: H204935

Symptom

When a failure is detected during early boot 'memory initialization' phase on a Dual In-Line Memory Module (DIMM), the Integrated Management Module (IMM) will log memory disabled as well as log scrub failure messages for that DIMM.

Note: This event is not a true Memory Scrub failure. This event is generated by the Unified Extensible Firmware Interface (UEFI) from a Power On Self Test (POST) Memory Failure, such as Failed Training or Failed MRC.

This feature was added across all IMM systems in the Third Quarter 2011 life cycle release and will be incorporated in IMM2. For example, when the system detects the memory error on slot 8, the log will have scrub failure in the log as well.

 

10 I 01/01/1970; 01:00:59 0x806f040c2008ffff "Memory Device 8" disabled on subsystem "System Memory"

11 E 01/01/1970; 01:00:59 0x806f030c2008ffff Scrub Failure for "Memory Device 8" on subsystem "System Memory"

Runtime memory failures that cannot be corrected will be logged with the following error:

Uncorrectable error detected for "Memory Device n" on subsystem "System Memory".(n = DIMM number)

SPECIAL Note:
See RETAIN Tip H211626, as it relates to reporting DIMM failures during POST time for some of the affected systems.

Affected configurations

The system can be any of the following IBM servers:

  • BladeCenter HS22, type 1936, any model
  • BladeCenter HS22, type 7870, any model
  • BladeCenter HS22V, type 1949, any model
  • BladeCenter HS22V, type 7871, any model
  • BladeCenter HS23, type 1929, any model
  • BladeCenter HS23, type 7875, any model
  • BladeCenter HS23E, type 8038, any model
  • BladeCenter HS23E, type 8039, any model
  • BladeCenter HX5, type 1909, any model
  • BladeCenter HX5, type 7872, any model
  • BladeCenter HX5, type 7873, any model
  • Flex System x220 compute node, type 2585, any model
  • Flex System x220 compute node, type 7864, any model
  • Flex System x220 compute node, type 7906, any model
  • Flex System x222 compute node, type 7916, any model
  • Flex System x240 compute node, type 7863, any model
  • Flex System x240 compute node, type 8737, any model
  • Flex System x240 compute node, type 8738, any model
  • System x3200 M3, type 7327, any model
  • System x3200 M3, type 7328, any model
  • System x3250 M3, type 4251, any model
  • System x3250 M3, type 4252, any model
  • System x3250 M3, type 4261, any model
  • System x3400 M2, type 7836, any model
  • System x3400 M2, type 7837, any model
  • System x3400 M3, type 7378, any model
  • System x3400 M3, type 7379, any model
  • System x3500 M2, type 7839, any model
  • System x3500 M3, type 7380, any model
  • System x3500 M4, type 7383, any model
  • System x3550 M2, type 4198, any model
  • System x3550 M2, type 7946, any model
  • System x3550 M3, type 4254, any model
  • System x3550 M3, type 7944, any model
  • System x3550 M4, type 7914, any model
  • System x3620 M3, type 7376, any model
  • System x3630 M3, type 7377, any model
  • System x3650 M2, type 4199, any model
  • System x3650 M2, type 7947, any model
  • System x3650 M3, type 4255, any model
  • System x3650 M3, type 7945, any model
  • System x3650 M4, type 7915, any model
  • System x3650 M4 BD, type 5466, any model
  • System x3690 X5, type 7147, any model
  • System x3690 X5, type 7148, any model
  • System x3690 X5, type 7149, any model
  • System x3690 X5, type 7192, any model
  • System x3750 M4, type 8722, any model
  • System x3750 M4, type 8733, any model
  • System x3850 X5, type 7143, any model
  • System x3850 X5, type 7145, any model
  • System x3850 X5, type 7146, any model
  • System x3850 X5, type 7191, any model
  • System x3950 X5, type 7143, any model
  • System x3950 X5, type 7145, any model
  • iDataPlex dx360 M2 server, type 6380, any model
  • iDataPlex dx360 M2 server, type 7321, any model
  • iDataPlex dx360 M2 server, type 7323, any model
  • iDataPlex dx360 M3 server, type 6391, any model
  • iDataPlex dx360 M4 2U Chassis, type 7913, any model
  • iDataPlex dx360 M4 server, type 7912, any model

This tip is not software specific.

This tip is not option specific.

The following system BIOS/UEFI levels are affected:

See the Additional Information section for affected code levels by system model and type.

The system has the symptom described above.

Workaround

The DIMM should be replaced according to the standard action plan for the memory scrub failure documented in the Problem Determination and Service Guide (PDSG) for the affected product.

Additional information

This behavior was changed in the Third Quarter 2011 release of IMM firmware. It is used to distinguish between POST and Runtime memory failures and ensures debug efforts can be focused in the right area. No UEFI change was required to enable this enhancement.

The error is reported by UEFI and notifies the IMM via the 'vgpio' command. Memory init code will disable DIMM(s) indicated by this error message. User should follow proper documented PDSG steps.

The following IMM firmware level(s) are affected:

  • BladeCenter HS22, HS22V and HX5 1.31 YUOOC7F or later
  • iDataPlex dx360 M2/M3 1.31 YUOOC7F or later
  • iDataPlex dx360 M4 1.25 1AOO26K or later
  • System x3200 M3 and x3250 M3 1.31 YUOOC7F or later
  • System x3400 M2/M3 and x3500 M2/M3 1.31 YUOOC7F or later
  • System x3500 M4 and x3550 M4 1.25 1AOO26K or later
  • System x3550 M2/M3 and x3650 M2/M3 1.31 YUOOC7F or later
  • System x3620 M3 and x3630 M3 1.08 HSE119AUS 1.31 YUOOC7F or later
  • System x3650 M4 1.26 1AOO26N or later
  • System x3530 M4 and x3630 M4 1.60 BEE136C or later
  • System x3690 X5 1.31 YUOOC7F or later
  • System x3850 X5 and x3950 X5 1.31 YUOOC7F or later

SPECIAL Note:
See RETAIN Tip H211626, as it relates to reporting DIMM failures during POST time for some of the affected systems.


Document Location

Worldwide

Operating System

BladeCenter:Operating system independent / None

System x:Operating system independent / None

PureFlex System and Flex System:Operating system independent / None

[{"Type":"HW","Business Unit":{"code":"BU016","label":"Multiple Vendor Support"},"Product":{"code":"QU01DEW","label":"System x->System x3500 M4->7383"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Line of Business":{"code":"","label":""}},{"Type":"HW","Business Unit":{"code":"BU016","label":"Multiple Vendor Support"},"Product":{"code":"QU01DKP","label":"System x->System x3650 M4->7915"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Line of Business":{"code":"","label":""}},{"Type":"HW","Business Unit":{"code":"BU056","label":"Miscellaneous"},"Product":{"code":"HW239","label":"BladeCenter HS23"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Line of Business":{"code":"LOB18","label":"Miscellaneous LOB"}},{"Type":"HW","Business Unit":{"code":"BU016","label":"Multiple Vendor Support"},"Product":{"code":"QU01GBW","label":"System x->System x3750 M4->8733"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Line of Business":{"code":"","label":""}},{"Type":"HW","Business Unit":{"code":"BU050","label":"BU NOT IDENTIFIED"},"Product":{"code":"QU01GQR","label":"PureFlex System and Flex System->x240 Compute Node->7863"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Line of Business":{"code":"","label":""}},{"Type":"HW","Business Unit":{"code":"BU054","label":"Systems w\/TPS"},"Product":{"code":"QU01GRH","label":"PureFlex System and Flex System->x240 Compute Node->8737"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Line of Business":{"code":"","label":""}},{"Type":"HW","Business Unit":{"code":"BU050","label":"BU NOT IDENTIFIED"},"Product":{"code":"QU01GRJ","label":"PureFlex System and Flex System->x220 Compute Node->7906"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Line of Business":{"code":"","label":""}},{"Type":"HW","Business Unit":{"code":"BU050","label":"BU NOT IDENTIFIED"},"Product":{"code":"QU02BWN","label":"PureFlex System and Flex System->x222 Compute Node->7916"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Line of Business":{"code":"","label":""}},{"Type":"HW","Business Unit":{"code":"BU016","label":"Multiple Vendor Support"},"Product":{"code":"QU03TEU","label":"System x->System x iDataPlex dx360 M2 server->7323"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Line of Business":{"code":"","label":""}},{"Type":"HW","Business Unit":{"code":"BU016","label":"Multiple Vendor Support"},"Product":{"code":"QU03VIF","label":"System x->System x iDataPlex dx360 M2 server->7321"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Line of Business":{"code":"","label":""}},{"Type":"HW","Business Unit":{"code":"BU056","label":"Miscellaneous"},"Product":{"code":"HW21Q","label":"BladeCenter HS22"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Line of Business":{"code":"LOB18","label":"Miscellaneous LOB"}},{"Type":"HW","Business Unit":{"code":"BU056","label":"Miscellaneous"},"Product":{"code":"HW21Q","label":"BladeCenter HS22"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Line of Business":{"code":"LOB18","label":"Miscellaneous LOB"}},{"Type":"HW","Business Unit":{"code":"BU016","label":"Multiple Vendor Support"},"Product":{"code":"QU03WCX","label":"System x->System x3650 M2->7947"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Line of Business":{"code":"","label":""}},{"Type":"HW","Business Unit":{"code":"BU016","label":"Multiple Vendor Support"},"Product":{"code":"QU03WKC","label":"System x->System x3550 M2->7946"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Line of Business":{"code":"","label":""}},{"Type":"HW","Business Unit":{"code":"BU054","label":"Systems w\/TPS"},"Product":{"code":"QU03WSY","label":"System x->System x iDataPlex dx360 M2 server->6380"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Line of Business":{"code":"","label":""}},{"Type":"HW","Business Unit":{"code":"BU054","label":"Systems w\/TPS"},"Product":{"code":"QU03WTQ","label":"System x->System x3550 M2->4198"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Line of Business":{"code":"","label":""}},{"Type":"HW","Business Unit":{"code":"BU054","label":"Systems w\/TPS"},"Product":{"code":"QU03WTS","label":"System x->System x3650 M2->4199"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Line of Business":{"code":"","label":""}},{"Type":"HW","Business Unit":{"code":"BU016","label":"Multiple Vendor Support"},"Product":{"code":"QU03XIC","label":"System x->System x3400 M2->7836"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Line of Business":{"code":"","label":""}},{"Type":"HW","Business Unit":{"code":"BU054","label":"Systems w\/TPS"},"Product":{"code":"QU03XIF","label":"System x->System x3400 M2->7837"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Line of Business":{"code":"","label":""}},{"Type":"HW","Business Unit":{"code":"BU016","label":"Multiple Vendor Support"},"Product":{"code":"QU03XIH","label":"System x->System x3500 M2->7839"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Line of Business":{"code":"","label":""}},{"Type":"HW","Business Unit":{"code":"BU016","label":"Multiple Vendor Support"},"Product":{"code":"QU04FAH","label":"System x->System x3200 M3->7328"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Line of Business":{"code":"","label":""}},{"Type":"HW","Business Unit":{"code":"BU056","label":"Miscellaneous"},"Product":{"code":"HW232","label":"BladeCenter HS22V"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Line of Business":{"code":"LOB18","label":"Miscellaneous LOB"}},{"Type":"HW","Business Unit":{"code":"BU056","label":"Miscellaneous"},"Product":{"code":"HW232","label":"BladeCenter HS22V"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Line of Business":{"code":"LOB18","label":"Miscellaneous LOB"}},{"Type":"HW","Business Unit":{"code":"BU054","label":"Systems w\/TPS"},"Product":{"code":"QU04IMI","label":"System x->System x3200 M3->7327"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Line of Business":{"code":"","label":""}},{"Type":"HW","Business Unit":{"code":"BU016","label":"Multiple Vendor Support"},"Product":{"code":"QU04INM","label":"System x->System x3250 M3->4251"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Line of Business":{"code":"","label":""}},{"Type":"HW","Business Unit":{"code":"BU016","label":"Multiple Vendor Support"},"Product":{"code":"QU04IOF","label":"System x->System x3250 M3->4252"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Line of Business":{"code":"","label":""}},{"Type":"HW","Business Unit":{"code":"BU054","label":"Systems w\/TPS"},"Product":{"code":"QU04IPR","label":"System x->System x3250 M3->4261"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Line of Business":{"code":"","label":""}},{"Type":"HW","Business Unit":{"code":"BU016","label":"Multiple Vendor Support"},"Product":{"code":"QU04SLL","label":"System x->System x3650 M3->7945"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Line of Business":{"code":"","label":""}},{"Type":"HW","Business Unit":{"code":"BU016","label":"Multiple Vendor Support"},"Product":{"code":"QU04SMA","label":"System x->System x3550 M3->7944"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Line of Business":{"code":"","label":""}},{"Type":"HW","Business Unit":{"code":"BU016","label":"Multiple Vendor Support"},"Product":{"code":"QU04SMO","label":"System x->System x iDataPlex dx360 M3 server->6391"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Line of Business":{"code":"","label":""}},{"Type":"HW","Business Unit":{"code":"BU016","label":"Multiple Vendor Support"},"Product":{"code":"QU04SNM","label":"System x->System x3400 M3->7378"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Line of Business":{"code":"","label":""}},{"Type":"HW","Business Unit":{"code":"BU054","label":"Systems w\/TPS"},"Product":{"code":"QU04SNO","label":"System x->System x3400 M3->7379"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Line of Business":{"code":"","label":""}},{"Type":"HW","Business Unit":{"code":"BU016","label":"Multiple Vendor Support"},"Product":{"code":"QU04SOK","label":"System x->System x3500 M3->7380"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Line of Business":{"code":"","label":""}},{"Type":"HW","Business Unit":{"code":"BU054","label":"Systems w\/TPS"},"Product":{"code":"QU04SPC","label":"System x->System x3550 M3->4254"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Line of Business":{"code":"","label":""}},{"Type":"HW","Business Unit":{"code":"BU016","label":"Multiple Vendor Support"},"Product":{"code":"QU04SPI","label":"System x->System x3650 M3->4255"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Line of Business":{"code":"","label":""}},{"Type":"HW","Business Unit":{"code":"BU016","label":"Multiple Vendor Support"},"Product":{"code":"QU04SRF","label":"System x->System x3850 X5->7146"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Line of Business":{"code":"","label":""}},{"Type":"HW","Business Unit":{"code":"BU016","label":"Multiple Vendor Support"},"Product":{"code":"HW233","label":"BladeCenter HX5"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Line of Business":{"code":"LOB57","label":"Power"}},{"Type":"HW","Business Unit":{"code":"BU016","label":"Multiple Vendor Support"},"Product":{"code":"QU04SRO","label":"System x->System x3850 X5->7145"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Line of Business":{"code":"","label":""}},{"Type":"HW","Business Unit":{"code":"BU016","label":"Multiple Vendor Support"},"Product":{"code":"QU04SZB","label":"System x->System x3950 X5->7145"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Line of Business":{"code":"","label":""}},{"Type":"HW","Business Unit":{"code":"BU016","label":"Multiple Vendor Support"},"Product":{"code":"SUNSET","label":"PRODUCT REMOVED"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Line of Business":{"code":"","label":""}},{"Type":"HW","Business Unit":{"code":"BU016","label":"Multiple Vendor Support"},"Product":{"code":"QU04WDX","label":"System x->System x3690 X5->7149"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Line of Business":{"code":"","label":""}},{"Type":"HW","Business Unit":{"code":"BU016","label":"Multiple Vendor Support"},"Product":{"code":"QU04WDY","label":"System x->System x3690 X5->7148"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Line of Business":{"code":"","label":""}},{"Type":"HW","Business Unit":{"code":"BU016","label":"Multiple Vendor Support"},"Product":{"code":"QU90ABB","label":"System x->System x3620 M3->7376"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Line of Business":{"code":"","label":""}},{"Type":"HW","Business Unit":{"code":"BU016","label":"Multiple Vendor Support"},"Product":{"code":"QU90ABC","label":"System x->System x3630 M3->7377"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Line of Business":{"code":"","label":""}},{"Type":"HW","Business Unit":{"code":"BU016","label":"Multiple Vendor Support"},"Product":{"code":"QU90ABO","label":"System x->System x3850 X5->7191"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Line of Business":{"code":"","label":""}},{"Type":"HW","Business Unit":{"code":"BU016","label":"Multiple Vendor Support"},"Product":{"code":"QU90ABQ","label":"System x->System x3690 X5->7147"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Line of Business":{"code":"","label":""}},{"Type":"HW","Business Unit":{"code":"BU016","label":"Multiple Vendor Support"},"Product":{"code":"QU90ABX","label":"System x->System x3850 X5->7143"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Line of Business":{"code":"","label":""}},{"Type":"HW","Business Unit":{"code":"BU016","label":"Multiple Vendor Support"},"Product":{"code":"QU90ACM","label":"System x->System x3690 X5->7192"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Line of Business":{"code":"","label":""}},{"Type":"HW","Business Unit":{"code":"BU056","label":"Miscellaneous"},"Product":{"code":"HW23F","label":"BladeCenter HS23E"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Line of Business":{"code":"LOB18","label":"Miscellaneous LOB"}},{"Type":"HW","Business Unit":{"code":"BU016","label":"Multiple Vendor Support"},"Product":{"code":"SUNSET","label":"PRODUCT REMOVED"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Line of Business":{"code":"","label":""}},{"Type":"HW","Business Unit":{"code":"BU016","label":"Multiple Vendor Support"},"Product":{"code":"QU90ADI","label":"System x->System x iDataPlex dx360 M4 server->7913"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Line of Business":{"code":"","label":""}},{"Type":"HW","Business Unit":{"code":"BU016","label":"Multiple Vendor Support"},"Product":{"code":"QU90ADT","label":"System x->System x3950 X5->7143"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Line of Business":{"code":"","label":""}},{"Type":"HW","Business Unit":{"code":"BU016","label":"Multiple Vendor Support"},"Product":{"code":"QU91IPI","label":"System x->System x3550 M4->7914"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Line of Business":{"code":"","label":""}},{"Type":"HW","Business Unit":{"code":"BU056","label":"Miscellaneous"},"Product":{"code":"HW239","label":"BladeCenter HS23"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Line of Business":{"code":"LOB18","label":"Miscellaneous LOB"}},{"Type":"HW","Business Unit":{"code":"BU016","label":"Multiple Vendor Support"},"Product":{"code":"QU91LBC","label":"System x->System x iDataPlex dx360 M4 2U chassis->7913"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Line of Business":{"code":"","label":""}},{"Type":"HW","Business Unit":{"code":"BU016","label":"Multiple Vendor Support"},"Product":{"code":"QU91NAJ","label":"System x->System x3750 M4->8722"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Line of Business":{"code":"","label":""}},{"Type":"HW","Business Unit":{"code":"BU056","label":"Miscellaneous"},"Product":{"code":"HW23F","label":"BladeCenter HS23E"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Line of Business":{"code":"LOB18","label":"Miscellaneous LOB"}},{"Type":"HW","Business Unit":{"code":"BU050","label":"BU NOT IDENTIFIED"},"Product":{"code":"QU91OLT","label":"PureFlex System and Flex System->x220 Compute Node->2585"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Line of Business":{"code":"","label":""}},{"Type":"HW","Business Unit":{"code":"BU054","label":"Systems w\/TPS"},"Product":{"code":"QUOEDBG","label":"PureFlex System and Flex System->x240 Compute Node->8738"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Line of Business":{"code":"","label":""}},{"Type":"HW","Business Unit":{"code":"BU016","label":"Multiple Vendor Support"},"Product":{"code":"QUOFE4B","label":"System x->System x3650 M4 BD->5466"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Line of Business":{"code":"","label":""}}]

Document Information

Modified date:
19 April 2023

UID

ibm1MIGR-5089246

Page Feedback