Stalls or unexpectedly reboots with 8k and SATA drives - IBM System x

Resolving The Problem

Source

RETAIN tip: H19856

Symptom

While running a heavy work load, the System x3650 or System x3655 servers could exhibit the following symptoms when configured with a ServeRAID-8K controller and SATA hard disk drives:

1. The system may stall occasionally for up to 30 seconds and then resume normal operations. As a result of the stall, command time-out event messages will be logged into the local ServeRAID Manager logs with the filename of RaidEvtA.log.
2. In severe cases, the server may experience an unexpected system restart. Depending on the operating system, the system may restart following a serious operating system related error condition, like a Microsoft Windows "BlueScreen" crash or a Linux kernel panic.

Affected configurations

The system may be any of the following IBM servers:

• System x3650 T, type 7980, any model
• System x3650, type 1914, any model
• System x3650, type 7979, any model
• System x3655, type 7985, any model

The system is configured with one or more of the following IBM Options:

• ServeRAID-8k SAS Controller, Option 25R8064

This tip is not software specific.

Solution

There are two firmware updates necessary to resolve this issue.

The first is a ServeRAID firmware update to build 15407 or higher. The second update is the system backplane firmware version 1.07. It is very important to update the ServeRAID controller firmware first, then apply the firmware update to the SAS/SATA hot swap backplane.

• ServeRAID8k and ServeRAID-8k-l firmware
• System x3650 and x3655 SAS/SATA hot-swap backplane firmware update program v1.07

Workaround

As an interim solution, the "PHY Rate" can be changed from 3.0 GB to 1.5 GB within the Adaptec Configuration Utility (ACU) of the ServeRAID-8k. Enter the ACU during the ServeRAID POST banner when prompted for CRTL-A.

To change the setting, complete the following steps within the ACU:

1. From the main menu of the ACU, select "Serial Select Utility".
2. Choose "PHY Configuration".
3. Observe the first row of values for the "PHY Rate" that defaults to "Auto".
4. Change the "PHY Rate" for channels #0, #1, #2, #3 to "1.5" by highlighting the row and column, and then pressing Enter to choose from the available options.
5. Press ESC to exit and save the changes when prompted.

Additional information

It has been determined that the SAS expander that bridges to SATA devices has a permanent limitation that prevents the expander from operating at 3.0 Gb speeds to SATA II devices.

The backplane firmware will change the PHY data rate to SATA devices to operate at 1.5 Gb. SAS hard disk drives will continue to operate properly at 3.0 Gb speeds.

Under "typical usage", there should only be a minor performance impact or no noticeable impact on performance. The term "typical usage" is defined as a system with a varying high, medium and low usage stress loads throughout a business day.

Non-typical usage is defined as a high stress load without variation for an extended period of time.

The order of installing the firmware updates is very important to avoid a possible hang condition. Always install the ServeRAID firmware update first, then apply the hot swap backplane firmware update. Updating to the latest ServeRAID firmware may also require that some other supporting software is also updated, e.g. ServeRAID device drivers, CLI tools and management software.