Cannot see NMI messages made by NMI button in RHEL4 and SLES9 - IBM System x

Resolving The Problem

Source

RETAIN tip: H187823

Symptom

The following Non-Maskable Interrupt (NMI) message is not shown in /var/log/message or the console when Red Hat Enterprise Linux 4 (RHEL4) Update 3 (U3) X86-64 is installed and the NMI button is pressed:

Uhhuh. NMI received for unknown reason 21 on CPU x

This issue also happens with SUSE Linux Enterprise Server (SLES9) Service pack 3 (SP3) X86-64.

Affected configurations

The system may be any of the following IBM systems:

• IntelliStation M Pro (M50), Type 9229, any model
• System x3105, Type 4347, any model
• System x3200, Type 4362, any model
• System x3200, Type 4363, any model
• System x3250, Type 4364, any model
• System x3250, Type 4365, any model
• System x3400, Type 7973, any model
• System x3400, Type 7974, any model
• System x3400, Type 7975, any model
• System x3400, Type 7976, any model
• System x3500, Type 7977, any model
• System x3550, Type 7978, any model
• System x3650 T, Type 7980, any model
• System x3650, Type 7979, any model
• System x3800, Type 8865, any model
• System x3850, Type 8863, any model
• System x3950 E, Type 8874, any model
• System x3950, Type 8872, any model
• xSeries 100, Type 8486, any model
• xSeries 135, Type 8654, any model
• xSeries 150, Type 8658, any model
• xSeries 200, Type 8478, any model
• xSeries 205, Type 8480, any model
• xSeries 206, Type 8482, any model
• xSeries 206m, Type 8485, any model
• xSeries 206m, Type 8490, any model
• xSeries 220, Type 8645, any model
• xSeries 225, Type 8647, any model
• xSeries 226, Type 8488, any model
• xSeries 226, Type 8648, any model
• xSeries 230, Type 8658, any model
• xSeries 232, Type 8668, any model
• xSeries 235, Type 8671, any model
• xSeries 236, Type 8841, any model
• xSeries 240, Type 8664, any model
• xSeries 250, Type 8665, any model
• xSeries 255, Type 8685, any model
• xSeries 260, Type 8865, any model
• xSeries 300, Type 8672, any model
• xSeries 305, Type 8673, any model
• xSeries 306, Type 8489, any model
• xSeries 306, Type 8836, any model
• xSeries 306m, Type 8491, any model
• xSeries 306m, Type 8849, any model
• xSeries 330, Type 8654, any model
• xSeries 335, Type 8676, any model
• xSeries 336, Type 1879, any model
• xSeries 336, Type 8837, any model
• xSeries 340, Type 8656, any model
• xSeries 342, Type 8669, any model
• xSeries 343, Type 8827, any model
• xSeries 345, Type 8670, any model
• xSeries 346, Type 1880, any model
• xSeries 346, Type 8840, any model
• xSeries 350, Type 8682, any model
• xSeries 360, Type 8686, any model
• xSeries 365, Type 8861, any model
• xSeries 366, Type 8863, any model
• xSeries 370, Type 8681, any model
• xSeries 380, Type 8683, any model
• xSeries 382, Type 8834, any model
• xSeries 440, Type 8687, any model
• xSeries 445, Type 8870, any model
• xSeries 450, Type 8688, any model
• xSeries 455, Type 8855, any model
• xSeries 460, Type 8872, any model
• xSeries Internet Appliances, Type 8645, any model
• xSeries MXE-460, Type 8874, any model
• xSeries RXE-100, Type 8684, any model

The system is configured with at least one of the following:

• Red Hat Enterprise Linux 4, any update, any Update
• SUSE Linux Enterprise Server 9, any Service Pack

This tip is not option specific.

The system has the symptom described above.

Note: This does not imply that the network operating system will work under all combinations of hardware and software. Please see the compatibility page for more information: http://www.ibm.com/servers/eserver/serverproven/compat/us/

Workaround

There is no Fix for this issue. This is a 64-bit Operating System (OS) feature by design and there are no plans to do any changes for this feature.

1. If the user wishes to see the NMI message, disable the NMI watchdog by appending the appropriate parameters at the boot prompt in RHLE4 and SLES9. This can be done by the following steps:
   
   RHLE4:
   
   a. When the blue splash screen is displayed, press any key to display the kernel menu.
   
   b. Move the cursor to a boot session, for example "2.6.9-34.ELsmp," and press 'e' to edit this session.
   
   c. Move the cursor to the line that starts with kernel, and press 'e' to edit that kernel line.
   
   d. Append 'nmi_watchdog=0' at the end of that kernel line.
   
   e. Press 'enter' to accept this change.
   
   f. Press 'b' to boot.
   
   SLES9:
   
   a. When the blue splash screen is displayed, press 'ESC' and 'OK' to leave the graphical boot menu and enter the text mode interface.
   
   b. Move cursor to a boot session, select "Linux", press 'e' to edit this session.
   
   c. Move cursor to the line starts with kernel, and press 'e' to edit that kernel line.
   
   d. Append 'nmi_watchdog=0' at the end of that kernel line.
   
   e. Press 'enter' to accept this change.
   
   f. Press 'b' to boot.
   
   
2. After the system boot up, complete the following steps to modify the configuration file of GNU GRUB:
   
   a. Log in as root and using vi (or any other editor), edit the /boot/grub/menu.lst file.
   
   b. Add 'nmi_watchdog=0' to the end of the kernel line.
   
   c. Type 'init 6' to restart the server.
   
   
3. When the above is finished, the user should have disabled NMI watchdog and should see the following from /var/log/message once the NMI button is pressed:
   
   Uhhuh. NMI received for unknown reason 21 on CPU x.

Additional information

A NMI button is usually used to generate a NMI signal to the processor and troubleshoot software and device driver errors when the system is experiencing problems that do not report any error information. In general, once this NMI button is pushed, user should see the report of NMI activity from /var/log/message or the console. However, the NMI message is not shown in /var/log/message or the console when you install RHEL4-U3 or SLES9 SP3 X86-64 and press the NMI button.

This is because RHEL4 Update 3 x86-64 and SLES9 SP3 x86-64 both enable NMI watchdog by default and provide crash log capability by NMI watchdog.

The Network Operating System (Operating System) vendors have redirected the NMI signal to be triggered by the NMI watchdog and when this happens (under severe system crash) a special message such as "Uhhuh. NMI received for unknown reason 21 on CPU x" will show in /var/log/message. The basic design idea is to exclude the need of a NMI button (to be pressed manually). However, with this feature, no message will show in /var/log/message, even when an NMI button is pressed, except the user specifically disabled NMI watchdog.

Use the workaround to disable this, and hence a traditional NMI button can be functional.