Troubleshooting Power issues - IBM BladeCenter (8677)

Resolving The Problem

RETAIN tip: H19326

Troubleshooting power issues on BladeCenter E, Type 8677.

This tip is for information purposes only.

The system may be any of the following IBM servers:

• BladeCenter E, Type 8677, any model

This tip is not software specific.
This tip is not option specific.
The system has the symptom described above.

None. This tip is for information only

None.

BladeCenter (Type 8677) Power Overview and Problem Determination:

All devices in the BladeCenter have a Light Emitting Diode (LED), which indicates that power is present. If a device will not boot, but shows indications of having power, refer to the troubleshooting documents for that device. In rare cases, devices that are inserted into the chassis will not show any indication of power being present. Use this document to troubleshoot those situations.

All blades and modules are powered by 12 volt DC which is distributed by the system midplane. The midplane is divided into four power segments with each segment sourced by a single power module.

To achieve redundancy, each blade/module has access to two of these 12 volt power segments (top or bottom supply) and will draw power from either or both sources. Modules and blades use diodes and FET switches to isolate the two power segments. Power control circuitry on the Modules/Blades performs over current detection and checks the integrity of the isolation circuits. The four power supplies are combined to create two power domains. The power domains distribute power to individual devices as follows:

Power Domain 1 = Power Module 1 (top) and 2 (bottom):

• Blades 1-6
• Both Management Modules (MM)
• All four switch slots
• Both blowers
• Media tray (the rear panel display card is powered by the Media Tray)

Power Domain 2 = Power Module 3 (top) and 4 (bottom):

• Blades 7-14

Other key points:

• A problem in the top power segment of a power domain will not effect the bottom power segment and vice versa.
• For safety reasons, the power supplies are turned off if the rear shuttle is disengaged from the main chassis. Thus, if the internal media tray cable (not the media tray itself) is disconnected or damaged then no power supplies will turn on and there will be no power to the chassis.
• DC to DC regulators on the blades and modules are used to convert 12 volt to the voltage needed by the logic (5, 3.3, 2.5, or 1.8 volt, etc). A short in the regulators or logic will shut down the blade and will be detected by the Baseboard Management Controller (BMC). The BMC will log this to the MM as a system board fault and illuminate the appropriate LightPath LEDs.
• Check for damaged power connectors when removing blades. If damaged pins are detected, care must be taken not to propagate the fault to the midplane or other components.
• For all power modules, if the AC LED is on, power is being delivered to the Power Supply. If the DC LED is on, the Power Supply is delivering 12 volt DC to the midplane.

Power module related symptoms and steps to troubleshoot them:

SYMPTOM 1: No LEDs on one I/O or blade in a power domain, but some I/O modules or blades in the domain are properly lit.

1. Confirm that all power modules have AC and DC LEDs on. If one or more DC LEDs are off, move to Symptom 2. If both AC and DC LEDs are off on any supply, move to Symptom 3.
2. Verify whether or not the device is working by attempting to exercise the device (boot a blade and look at it via the remote control, insert a CD into a media tray and try to read it, etc.) If the device is working, one or more of the LEDs for the device (e.g. front bezel on a blade) is faulty. Contact IBM support to replace the faulty LED.
3. If the device is not working, remove it and check for loose connectors. Replace the device if loose connectors exist.
4. If no loose connectors are found, then swap the failing device with a working device of the same type. If the problem follows the device, then contact IBM support for possible replacement.
5. If the problem above stays with the slot, consult the document “Troubleshooting the 8677 chassis.” (chassis checkout)

SYMPTOM 2: One or more power modules have the AC LED on, but DC LED off.

1. Look at the Management Module (MM) Event Log for possible power related error messages. If the Event Log shows a error for a power module, contact IBM support.
2. If no errors are in the Event Log, and only one power module has AC LED on and DC LED off, swap this power module with another module that is working correctly.
   Note: Swapping power modules between power domain 1 and 2 may throttle or power off blades, depending on the number and type of blades you have in the chassis and the power policy selected in Fuel Gauge. Do not swap the power modules within the same domain unless you are willing for all devices in the chassis to reboot. If the symptom follows the power module, then contact IBM support for a possible power module replacement.
3. If the symptom stays with a particular slot, or multiple power modules have AC LED on and DC LED off, then remove the SPC chassis (also called the "rear shuttle") and reinsert it. Instructions for removing the SPC chassis are in Chapter 5 of the 8677 Hardware Maintenance Manual and Troubleshooting Guide. If that does not resolve the symptom, then contact IBM support.

SYMPTOM 3: Both AC and DC LEDs are off for multiple power modules.

1. Verify that the power modules are connected to 220 volt AC source. The AC LEDs will not turn on if the power source is 110 volt.
2. Reseat the AC power cords and modules that have both LEDs off.
3. Swap the AC power cord with a known good cord.
4. Check the AC sockets on the PDU to confirm that they are working.
5. Contact IBM BladeCenter support.