Troubleshooting media tray problems - IBM BladeCenter (Type 8677)

Resolving The Problem

RETAIN tip: H19909

How to troubleshoot media tray problems in the BladeCenter (Type 8677).

The system may be any of the following IBM servers:

• BladeCenter Chassis, Type 8677, any model

This Tip is not software specific.

This Tip is not option specific.

BLADECENTER (TYPE 8677) MEDIA TRAY TROUBLESHOOTING GUIDE

BladeCenter (Type 8677) Hardware Maintenance Manual and Troubleshooting Guide

System Service Parts - IBM BladeCenter (Type 8677 and 1881)

The file will be available from the IBM System x Support web site, at the following URL:

http://www.ibm.com/systems/support/

Use this procedure if a specific media tray function, such as front or rear panel LEDs, USB, CDROM or the diskette drive, is not working. Also use it if the BladeCenter chassis is experiencing a I2C Bus 4 or SP COMM error that affects multiple blade servers.

The BladeCenter chassis contains a hot-pluggable USB attached component called the media tray which contains the front system LED panel and the CDROM and diskette drives which are shared by all of the blades in the chassis. The media tray slides into a horizontal drawer along the top of the chassis and can be removed by pressing in on catches at both ends of the tray. Components of the media tray include the LED panel, CDROM, diskette drive, Customer Interface Card (CIC), diskette drive to CIC cable, media tray to CIC cable, CDROM interposer card, CDROM to CIC signal cable and CDROM to CIC power cable. Another part, called the Media Cable with Bracket, connects the media tray to the chassis midplane. Proper isolation of what is and what is not working in the media tray is essential to be able to identify which of these components might be causing the problem.

The front system LED panel is useful for getting a quick view of the health of the chassis. The green power-on LED indicates when AC power is applied to the chassis and that a functioning Management Module (MM) is present. The blue location LED can be turned on through the MM to identify a particular chassis in a rack. The amber over-temperature LED is lit when either the chassis or an individual blade server has detected a temperature level exceeded condition. The amber information LED is lit for non-critical events such as an incompatible I/O module plugged in the chassis or a chassis power demand exceeded condition. The amber system error LED is lit when a critical error has been detected for any pluggable module or blade in the chassis. There is a similar LED status panel on the rear of the chassis which is also controlled by the MM. More detail for an informational or a system error event is posted in the MM event log.

All media tray functions are controlled by the MM. The connection between USB devices in the media tray and blade servers is also controlled by the MM. Only one blade server can have access to a media tray USB device at a time and the ability to manually select which blade is connected to the media tray device using the blade front panel button can be disabled with a MM browser setting. Always check this MM setting first if the CD selection button on the front of the blade server does not appear to be working. If you are having trouble booting the blade to a media tray device then check the boot sequence for the blade using the MM browser (Blade Tasks - Configuration - Boot Sequence) to make sure the CDROM and/or diskette device is listed. A hardware or firmware defect present in the MM can also cause the media tray to stop functioning.

There is a dedicated serial control interface, aka I2C bus, connecting the MM to the media tray. An I2C interface is used for media tray presence detection and for control of media tray functions such as the front system LED panel. The same I2C bus is also connected to the rear panel LED controller and the midplane. Each MM in the chassis has it's own I2C bus connection to the media tray, for redundancy and fault tolerance. So, if the media tray is not functioning properly because of a bad I2C connection then either switching chassis control to the redundant MM or moving the MM from bay 1 to bay 2 will probably isolate the problem. There are several I2C control buses running throughout the chassis. Any critical failures on this media tray bus will show up as a "Failure reading I2C device. Check devices on bus 4" error message in the MM event log.

The system LED panel control circuit and the USB device hub are located on a card in the media tray called the Customer Interface Card or CIC. As mentioned earlier, several cables are used to connect media tray components to this card. Always check the cable and interposer card connections first before suspecting other active components in the media tray, blades or MM. There are two USB ports, A and B, that route from this card through the chassis midplane to all of the blades. Firmware in the MM controls media device selection by issuing commands to the blade service processor via an RS485 interface. The MM enables blade connection to the media tray through USB A or USB B. The blade service processor activates the control lines to the midplane to switch the USB buses to the blade. Only one blade may be connected to the media tray USB buses at a time.

Other than disconnecting the system LED panel and the chassis temperature sensor, removing the media tray will have no more effect on active blade servers than unplugging any USB device from a server. As long as the OS is not in the process of accessing one of the USB devices in the media tray then the server will continue to function normally if the media tray is removed. The chassis fans will ramp up to 100% but this is normal. Removal of the media tray is a good troubleshooting technique to isolate I2C bus communication problems.

1. Diagnosing error symptoms - IBM eServer BladeCenter (Type 8677).
2. Troubleshooting CD and DVD drive issues - Servers and IntelliStation.
3. Troubleshooting media tray issues - IBM BladeCenter.
4. CD-ROM drive does not appear in the resource list - IBM BladeCenter JS20.
5. "CD not found" error during Red Hat Enterprise Linux (RHEL) 4 installation on blades.
6. Troubleshooting USB issues - Servers and IntelliStation.
7. SP COMM Errors in the MM event log

A. Initial Checkout And Problem Isolation:

1. Login to the MM browser and check the MM event log. Are there SP COMM errors in the log for multiple blades? If so, call IBM Support and follow the instructions in IBM Retain Tip # H184430 to verify that the Chassis is at the correct EC level.
2. Is this problem affecting more than one blade in the chassis? Verify that local KVM control is enabled in the MM browser by looking under Blade Tasks - Remote Control. Scroll to the bottom of this screen and verify Disable local KVM switching and Disable local Media Tray Switching are not checked. Uncheck these options if checked and click on Save.
3. Check the System Status page in the MM browser to make sure that all the blades have been discovered. Media Tray selection will not work properly for any blade that still shows "Discovering" status.
4. Check the Power LED on the blade front bezel. Does it blink when the blade is powered off and turn on solid when the blade is powered on? If this LED does not light up at all then first try replugging the blade front bezel cable then replace the blade front bezel.
5. At this point we know the blade power LED lights up solid and the blade powers up. Does the CD LED on the blade front bezel light up solid when pressed? If the LED blinks instead, wait 30 seconds to give the MM time to communicate with the blade. If the LED still doesn't light up solid then there might be a communication problem between the blade and the MM. Press the CD button for another blade in the same chassis to see if it lights up ok. If the CD button fails for more then one blade then go to Section B.

   Note: It is possible to get ahead of the MM when pressing media selection buttons between the blades. Always allow a few seconds for the MM to respond before selecting another blade.

6. At this point we know that only this one blade has a problem with the CD media selection button. We need to isolate whether the bug is in the blade or the chassis. If possible, swap this blade with another blade in the chassis that works or just move the blade to an empty slot in the same chassis. If you cannot move the blade to another slot then try replugging the blade to reset the service processor. If this doesn't fix it, call IBM BladeCenter support.
   
   
   • If the CD button problem follows the blade, then suspect the blade. Check the current BMC Service Processor firmware level by looking in the MM Firmware VPD page. Look up the BMC firmware change history for this blade type on the IBM support site and look for errata that might affect your configuration. Call IBM BladeCenter support.
   • If the CD button problem stays with the slot, move the blade back to the original slot and try again. If it still doesn't work then refer to the "Chassis Checkout troubleshooting procedure".
     
     
7. Continue here if the CD selection button appears to be working. Is this a media access problem where the blade cannot boot or read data from a device in the media tray? If no, go to "Media Tray Communication Problems". If yes, which media device is it, CDROM, diskette or external USB? If the problem is isolated to just the CDROM or the diskette drive, then try a different piece of media, i.e. burn another CD (If it s a DVD, verify the optical drive in this chassis supports DVD media). Verify the CD or diskette media works with another blade server in the chassis. Verify the correct USB device driver is loaded for the OS running on the blade. If it's a boot problem, make sure the appropriate entry for the CDROM or diskette drive is listed in the blade boot sequence, shown in the MM under Blade Tasks - Configuration - Boot Sequence.
8. If the symptom is still pointing to either the CDROM or diskette drive in the media tray and this is the only blade showing the symptom, then suspect the blade or the slot. If possible, move the blade to a known good slot in the chassis (one that had a blade in it that accessed either the CDROM or diskette drive ok).
   
   
   • If the blade still can not access either the CDROM or diskette drive after moving it to another slot and you know the OS device driver is good, then replace the blade system board.
   • If after moving the blade to another slot it now accesses either the CDROM or diskette drive ok then the chassis may have a slot problem. Refer to the "Chassis Checkout procedure".
     
     
9. If the problem is with an external USB device, verify that the device is compatible with the BladeCenter USB interface and the software running on the blade. The BladeCenter (Type 8677) external USB interface accepts devices which are compatible with USB specification 1.1. Check the compatibility information for the external USB device and make sure the blade Operating System type and version is supported. Verify the latest device driver is installed. Try the USB device on a non-blade server or blade server in another chassis running the same Operating System type and version. If the device works on another server and you have confirmed that the software drivers are good then.
   
   
   • Either the device is incompatible with the chassis USB port, probably because of a power overcurrent condition, or the USB port is bad. Try a different type of USB device in this port to see it if works.
   • If no external USB devices work in the USB port, replace the customer interface card, CIC, in the media tray.

B. Media Tray Problem Affecting Multiple Blades Or A Blade In Multiple Slots:

1. Remove the media tray (this will not affect normal blade operation), open the tray top cover, check and reseat all of the cable and drive connections. Check the connector at the rear of the media tray and use a flashlight to check the corresponding connector in the chassis for a broken socket or bent pin. If a bent pin is found then call IBM BladeCenter support, otherwise reinstall the media tray.
2. Are there SP COMM errors in the MM event log for more than one blade? If so, then refer to the MM Connectivity troubleshooting guide to debug the MM. Continue this procedure after verifying that it's not a bad MM or MM connectivity issue.
3. If this is a media access problem then first find out if both the CDROM and the diskette drive are affected. If the blades cannot access either device then suspect the parts that are common to both, i.e. the CIC, media tray to CIC cable and the media cable with bracket. The media cable with bracket can only be inspected by taking down the entire chassis and removing the SPC chassis and midplane, so it should be treated last. Replace media tray parts and retest in this order:
   
   
   • Replace CIC, install media tray, then retest.
   • Replace media tray to CIC cable, install media tray, then re-test.
   • Shutdown all of the blades, power down the chassis, replace the Media Cable with Bracket, reinstall the midplane, the SPC chassis with only one power supply in bay 1, one MM and one blower, one blade in slots 1-6 and the media tray. Test media access from the blade to the media tray.
     
     
4. If the blades can access one device but not the other, e.g. can access the CDROM but not the diskette, then suspect parts unique to that device. First inspect the cables again looking for obvious wiring breaks and pin or socket problems. For the CDROM, replace the drive, then the interposer card then the CDROM to CIC power and signal cables (if the CDROM LED blinks at some point, then the power cable is good). For the diskette, replace the drive then the diskette drive to CIC cable.

C. Media Tray Communications Problems

1. If this is not a media access problem, check the MM event log for I2C errors for devices on bus 4. If you see an error like this in the event log, follow the repair procedure identified in the 8677 HMM and Troubleshooting Guide for "Failure reading I2C device. Check devices on bus 4."
2. If there are no error messages in the event log then follow this procedure:
   
   
   • If the front panel LED's are not working, then replace the media tray CIC.
   • If both front and rear panel LED's are not working, then replace the media tray CIC.
   • If the front panel LED s are working but the rear panel LED's are not working, then replace the rear panel interface card.
     
     
3. If the media tray is still not working properly refer to the I2C bus troubleshooting procedure.

E. Chassis Checkout

1. This procedure is in another BladeCenter troubleshooting document. Call IBM BladeCenter Support for more information.

None.

None.