Troubleshooting
Problem
There are number of advanced features implemented in the memory subsystem of IBM's BladeCenter Architecture which actively monitor Dual In-Line Modules (DIMMs). If memory errors are detected, the BladeCenter system will log one (1) of the following events: 0x806F010C - uncorrectable ECC memory error 0x806F030C - memory scrub failed (Power On Self Test (POST) MRC/Training error) 0x806F050C - correctable ECC memory error logging limit reached (where: ECC = Error Correction Code) In addition, to maintain the highest levels of system availability, if a memory error is detected during POST or memory configuration, the server can disable the memory bank containing the failed memory DIMM automatically and continue operating with reduced memory capacity.
Resolving The Problem
Source
RETAIN tip: H21455
Symptom
There are number of advanced features implemented in the memory subsystem of IBM's BladeCenter Architecture which actively monitor Dual In-Line Modules (DIMMs). If memory errors are detected, the Advanced Management Module (AMM) will log one of the following events. Similar messages will be logged in the IMM/IMMv2 event log of the server:
| 0x806F010C - uncorrectable ECC
memory error 0x806F030C - memory scrub failed (Power On Self Test (POST) MRC/Training error) 0x806F050C - correctable ECC memory error logging limit reached (where: ECC = Error Correction Code) |
Respectively System UEFI or POST diagnostic error codes will be generated when the server starts up or while the server is running and the memory error is detected:
|
[W.58001] The PFA Threshold limit (correctable error logging limit) has been exceeded on DIMM number percent at address percent. MC5 Status contains percent and MC5 Misc contains percent. [S.51003] An uncorrectable memory error was detected in DIMM slot percent on rank percent. [S.51003] An uncorrectable memory error was detected on processor percent channel percent. The failing DIMM within the channel could not be determined. [S.51003] An uncorrectable memory error has been detected during POST. [S.58008] A DIMM has failed the POST memory test. |
In addition, to maintaining the highest levels of system availability, if a memory error is detected during Power On Self Test (POST) or memory configuration, the server can disable the memory bank containing the failed memory DIMM automatically and continue operating with reduced memory capacity.
Affected configurations
The system may be any of the following IBM servers:
- BladeCenter HS22, type 1911, any model
- BladeCenter HS22, type 1936, any model
- BladeCenter HS22, type 7809, any model, any any
- BladeCenter HS22, type 7870, any model
- BladeCenter HS22V, type 1949, any model
- BladeCenter HS22V, type 7871, any model
- BladeCenter HS23, type 1929, any model
- BladeCenter HS23, type 7875, any model
- BladeCenter HS23, type 7875 E5-xxxxV2, any model
- BladeCenter HS23E, type 8038, any model
- BladeCenter HS23E, type 8039, any model
- BladeCenter HX5, type 1909, any model
- BladeCenter HX5, type 1910, any model
- BladeCenter HX5, type 7872, any model
- BladeCenter HX5, type 7873, any model
This tip is not software specific.
This tip is not option specific.
The following system BIOS or UEFI level(s) are affected: Refer to fix section for affected list of FW
Workaround
Following are the minimum system firmware build levels that should be running on an IBM BladeCenter servers prior to replacement of memory DIMMs for memory scrub and correctable/uncorrectable ECC memory error events:
| BladeCenter HS23 imm2_1aoo50c-3.60
uefi_tke136v-1.50 BladeCenter HS23E imm2_1aoo50d-3.65 uefi_ahe136a-1.40 BladeCenter HX5 imm_yuoof7c-1.41 uefi_hie179b-1.79 BladeCenter HS22 imm_yuoof7c-1.41 uefi_p9e159a-1.20 BladeCenter HS22V imm_yuoof7c-1.41 uefi_p9e159a-1.20 |
(where: IMM = Integrated Management Module)
(where: UEFI = Unified Extensible Firmware Interface)
The file is available by selecting the appropriate Product Group, type of System, Product name, Product machine type, and Operating system on IBM Support's Fix Central web page, at the following URL:
- After the latest system firmware build is applied to the blade server the user should re-enable the memory bank manually using the F1 Setup menu in the UEFI if DIMMs where disabled due to memory errors.
Refer to IBM BladeCenter Information Center or Problem Determination and Service Guide (PDSG) for steps related to the specific blade server that should be followed when taking actions for device specific events:
IMPORTANT: It should be noted that updating system firmware to the code levels listed in this document may not resolve all the memory subsystem issues but should address all memory issues known by IBM.
Additional information
System Unified Extensible Firmware versions listed in this document contain memory reference code updates as well as memory refresh and threshold optimization for predictive failure alerts (PFA), which will result in significant reduction in the rate of superfluous PFA alerts.
Document Location
Worldwide
Was this topic helpful?
Document Information
Modified date:
18 April 2023
UID
ibm1MIGR-5093251