SAS Expander command timeout when running consistency check

Troubleshooting

Problem

The operating system becomes unresponsive and records a command timeout in the logs when running a consistency check. The Serial Attached SCSI (SAS) controller firmware log (available in Dynamic System Analysis (DSA)) will report the command timeout ofthe SAS hard drives with the IBM ServeRAID M5014/M5015 SAS or SATA controller. An example of the events are shown below:

Resolving The Problem

Source

RETAIN tip: H203514

Symptom

The operating system becomes unresponsive and records a command timeout in the logs when running a consistency check.

The Serial Attached SCSI (SAS) controller firmware log (available in Dynamic System Analysis (DSA)) will report the command timeout of the SAS hard drives with the IBM ServeRAID M5014/M5015 SAS or SATA controller.

An example of the events are shown below:

	xx/xx/xx xx:xx:xx: EVT#xxxxxx-xx/xx/xx xx:xx:xx: 113=Unexpected sense: PD 19(e0x18/s0) Path 5005076028c160e9, CDB: 2a 00 00 2a 2a 81 00 00 08 00, Sense: 6/29/00 xx/xx/xx xx:xx:xx: Raw Sense for PD 18: 70 00 06 00 00 00 00 0a 00 00 00 00 29 00 00 00 00 00 xx/xx/xx xx:xx:xx: DM_PerformSpinupRecoverySM:: Exceeded Timeout Count. pRdm=806bd800 TimeoutCount=2 devId=19 RecState=2 xx/xx/xx xx:xx:xx: EVT#xxxxxx-xx/xx/xx xx:xx:xx: 87=Error on PD 18(e0x18/s0) (Error 02) xx/xx/xx xx:xx:xx: EVT#xxxxxx-xx/xx/xx xx:xx:xx: 114=State change on PD 19(e0x18/s0) from ONLINE(18) to FAILED(11) xx/xx/xx xx:xx:xx: EVT#xxxxxx-xx/xx/xx xx:xx:xx: 81=State change on VD 00/3 from OPTIMAL(3) to OFFLINE(0)

Affected configurations

The system may be any of the following IBM servers:

System x3400 M2, type 7836, any model
System x3400 M2, type 7837, any model
System x3400 M3, type 7378, any model
System x3400 M3, type 7379, any model
System x3500 M2, type 7839, any model
System x3500 M3, type 7380, any model
System x3650 M2, type 4199, any model
System x3650 M2, type 7947, any model
System x3650 M3, type 4255, any model
System x3650 M3, type 7945, any model
System x3690 X5, type 7147, any model
System x3690 X5, type 7148, any model
System x3690 X5, type 7149, any model
System x3690 X5, type 7192, any model

The system is configured with one or more of the following IBM Options:

ServeRAID Expansion Adapter, Option part number 60Y0309, replacement part number 46M0997
ServeRAID M1015 SAS or SATA Controller, Option part number 46M0831, replacement part number 46M0861
ServeRAID M5014 SAS or SATA Controller, Option part number 46M0916, replacement part number 46M0918
ServeRAID M5015 SAS or SATA Controller, Option part number 46M0829, replacement part number 46M0851 - Adapter

This tip is not software specific.

The 510A firmware for the ServeRAID Expansion Adapter is affected.

Solution

This behavior will be corrected in a future release of firmware version 602a for the ServeRAID Expansion Adapter, Option part number 60Y0309.

The target date for this release is scheduled for third quarter 2011.

The file is or will be available by selecting the appropriate Product name, Product machine type, and operating system on IBM Support's Fix Central web page, at the following URL:

http://www.ibm.com/support/fixcentral/systemx/groupView?query.productGroup=ibm%2FSystemx

Additional information

This issue, seen in firmware version 510A where the expander would occasionally not respond to an SSP command, is caused by an error in the SSP interrupt servicing routine (ISR) that is run every time an SSP interrupt occurs. There is a small timing window (less than one microsecond) where, if two different interrupts occurred at the same time (for example, the 5ms timer and the SSP interrupt), then only one of them is serviced. This results in SSP commands not being serviced.

Document Location

Worldwide

Operating System

System x:Operating system independent / None

System x Hardware Options:Operating system independent / None

Applicable countries and regions

Worldwide

[{"Type":"HW","Business Unit":{"code":"BU016","label":"Multiple Vendor Support"},"Product":{"code":"QU03WCX","label":"System x->System x3650 M2->7947"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Line of Business":{"code":"","label":""}},{"Type":"HW","Business Unit":{"code":"BU054","label":"Systems w\/TPS"},"Product":{"code":"QU03WTS","label":"System x->System x3650 M2->4199"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Line of Business":{"code":"","label":""}},{"Type":"HW","Business Unit":{"code":"BU016","label":"Multiple Vendor Support"},"Product":{"code":"QU03XIC","label":"System x->System x3400 M2->7836"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Line of Business":{"code":"","label":""}},{"Type":"HW","Business Unit":{"code":"BU054","label":"Systems w\/TPS"},"Product":{"code":"QU03XIF","label":"System x->System x3400 M2->7837"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Line of Business":{"code":"","label":""}},{"Type":"HW","Business Unit":{"code":"BU016","label":"Multiple Vendor Support"},"Product":{"code":"QU03XIH","label":"System x->System x3500 M2->7839"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Line of Business":{"code":"","label":""}},{"Type":"HW","Business Unit":{"code":"BU016","label":"Multiple Vendor Support"},"Product":{"code":"QU04SLL","label":"System x->System x3650 M3->7945"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Line of Business":{"code":"","label":""}},{"Type":"HW","Business Unit":{"code":"BU016","label":"Multiple Vendor Support"},"Product":{"code":"QU04SNM","label":"System x->System x3400 M3->7378"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Line of Business":{"code":"","label":""}},{"Type":"HW","Business Unit":{"code":"BU054","label":"Systems w\/TPS"},"Product":{"code":"QU04SNO","label":"System x->System x3400 M3->7379"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Line of Business":{"code":"","label":""}},{"Type":"HW","Business Unit":{"code":"BU016","label":"Multiple Vendor Support"},"Product":{"code":"QU04SOK","label":"System x->System x3500 M3->7380"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Line of Business":{"code":"","label":""}},{"Type":"HW","Business Unit":{"code":"BU016","label":"Multiple Vendor Support"},"Product":{"code":"QU04SPI","label":"System x->System x3650 M3->4255"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Line of Business":{"code":"","label":""}},{"Type":"HW","Business Unit":{"code":"BU016","label":"Multiple Vendor Support"},"Product":{"code":"QU04WDX","label":"System x->System x3690 X5->7149"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Line of Business":{"code":"","label":""}},{"Type":"HW","Business Unit":{"code":"BU016","label":"Multiple Vendor Support"},"Product":{"code":"QU04WDY","label":"System x->System x3690 X5->7148"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Line of Business":{"code":"","label":""}},{"Type":"HW","Business Unit":{"code":"BU016","label":"Multiple Vendor Support"},"Product":{"code":"QU90ABQ","label":"System x->System x3690 X5->7147"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Line of Business":{"code":"","label":""}},{"Type":"HW","Business Unit":{"code":"BU016","label":"Multiple Vendor Support"},"Product":{"code":"QU90ACM","label":"System x->System x3690 X5->7192"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Line of Business":{"code":"","label":""}}]

Was this topic helpful?

Document Information

More support for:
System x->System x3650 M2->7947

Document number:
859154

Modified date:
30 January 2019

UID

ibm1MIGR-5088500

IBM Support

Tips

SAS Expander command timeout when running consistency check - IBM System x