Troubleshooting
Problem
After a Flex System Management (FSM) node appliance update, the system reboots but fails to come back up. By inspecting the local console with the use of a keyboard-video-mouse (KVM) cable, or through the Integrated Management Module (IMM) remote viewer,the following error message is observed:
dracut Warning: Boot has failed. To debug this issue add "rdshell" to the kernel command line.
dracut Warning: Signal caught!
dracut Warning: Boot has failed. To debug this issue add "rdshell" to the kernelcommand line.
Kernel panic - not syncing: Attempted to kill init!
Pid: 1, comm: init Not tainted 2.6.32-431.5.1.el6.x86_64 #1
Call Trace:
[
Resolving The Problem
Source
RETAIN tip: H213401
Symptom
After a Flex System Management (FSM) node appliance update, the system reboots but fails to come back up. By inspecting the local console with the use of a keyboard-video-mouse (KVM) cable, or through the Integrated Management Module (IMM) remote viewer, the following error message is observed:
dracut Warning: Boot has failed. To debug this issue add "rdshell" to the kernel command line.
dracut Warning: Signal caught!dracut Warning: Boot has failed. To debug this issue add "rdshell" to the kernel command line.
Kernel panic - not syncing: Attempted to kill init!Pid: 1, comm: init Not tainted 2.6.32-431.5.1.el6.x86_64 #1
Call Trace:
[<ffffffff81527513>] ? panic+0xa7/0x16f[<ffffffff81077622>] ? do_exit+0x862/0x870
[<ffffffff8118a855>] ? fput+0x25/0x30
[<ffffffff81077688>] ? do_group_exit+0x58/0xd0
[<ffffffff81077717>] ? sys_exit_group+0x17/0x20
[<ffffffff8100b072>] ? system_call_fastpath+0x16/0x1b
Affected configurations
The system may be any of the following IBM servers:
- Flex System Manager Node, type 7955, any model
- Flex System Manager Node, type 8731, any model
- Flex System Manager Node, type 8734, any model
This tip is not software specific.
This tip is not option specific.
Solution
This kernel panic is caused by two (2) different partitions having the same device Label 'Root'. Use the following procedure to boot the FSM to single user mode and set the label name of device /dev/sdb3 to 'Backup'.
- Attach to the local console via either the use of a KVM cable
or by launching the IMM remote viewer.
- Power on the FSM and wait for the
RHEV-H bootloader screen.
- When the bootloader shows, press the space bar
to interrupt the boot process.
- RHEV-H should be selected, press the
'e' key. This will launch the boot option edit
menu.
- Select the second line (starts with
'kernel', press the 'e' key to
edit the line).
- Add kernel parameter "single" to the end of
the line.
- Use the left arrow key to move all the way to the beginning of
the line.
- Edit the kernel boot parameter that reads
'root=live:LABEL=Root' and change it to
'root=live:/dev/sda3'.
- The beginning of the line should look like this: kernel
/vmlinuz0 root=live:/dev/sda3 ro rootfstype=auto
rootflags[...]
- Do not make any other changes to the boot parameters.
- Press [Enter] to save the changes.
- Back on the GNU GRUB menu, press the
'b' key to boot the machine.
- The system will boot to a Linux command shell.
- Verify the Label names by issuing the 'blkid'
command. There should be two (2) different partitions with the same
'Root' label.
- Change the Label for device /dev/sdb3 to
'Backup' by issuing: e2label
/dev/sdb3 "Backup"
- Restart the FSM with the
'reboot' command.
- The FSM should boot normally.
Additional information
During an FSM upgrade procedure, if the firmware update portion of the FSM fails, it is possible the FSM is left in a state where two (2) partitions have the same label 'Root'.
The fix sections documents how to set the partitions to the correct naming conventions, allowing the FSM to boot up.
Document Location
Worldwide
Was this topic helpful?
Document Information
Modified date:
16 May 2022
UID
ibm1MIGR-5096477