Topic
  • 2 replies
  • Latest Post - ‏2014-03-21T06:31:58Z by Novikov_Alexander
chris8135
chris8135
2 Posts

Pinned topic x3650 m2 mr10i after update with bomc backplane slots don't matches drive ids

‏2014-03-03T00:36:51Z |

Hello,

I've 5 x3650 m2 servers here, pulled from dc after working 3y, I want to use it now for other tasks. I've ordered new drives, new cpus, upgrades and new ram to put it to a new level and upgrade all to strong servers with the same power like x3650 m3 which I also owns. Worked all fine until I did an upgrade of all with update cd created with bomc. Unfortunately I never checked the raid controller, until I tried to add an sas expansion card to add 4 more drives to one of it then I noticed that the backplane id's don't matches the physical drive id's. First I thought the sas expander from the upgrade kit is the faulty part, so I've removed it, same error in webbios. I checked the other 4, on all servers the same :-(

I used the latest serveraid formware from ibm site, I tried downgrades, all possible solutions failed. So I decided to use the latest lsi firmware form lso site:

[root@flash ~]# /opt/MegaRAID/MegaCli/MegaCli64  -adpfwflash -f mr1078fw.rom -a0

Adapter 0: IBM ServeRAID-MR10i SAS/SATA Controller
Vendor ID: 0x1000, Device ID: 0x0060

Package version on the controller: 11.0.1-0042
Package version of the image file: 11.0.1-0054
Download Completed.
Flashing image to adapter...
Adapter 0: Flash Completed.
Please reboot the system for the changes to take effect

Exit Code: 0x00

Still wrong ID's. Any Idea whats wrong? I found another user here with the same problem: https://www.ibm.com/developerworks/community/forums/html/topic?id=77777777-0000-0000-0000-000014893499#77777777-0000-0000-0000-000014898797

I can't use the servers this way, if a drives fails and somebody other changes it and pull the wrong than the array will be distroyed...

Test Server (8 HDD backplane, no expander) has the following test cfg:

0 - 146GB
1 - 146GB
2 - 146GB
7 - 73GB

This is what I get:

[root@flash ~]# /opt/MegaRAID/MegaCli/MegaCli64  -PDList -a0

Adapter #0

Enclosure Device ID: 252
Slot Number: 3
Drive's position: DiskGroup: 0, Span: 0, Arm: 0
Enclosure position: N/A
Device Id: 9
WWN:
Sequence Number: 2
Media Error Count: 0
Other Error Count: 0
Predictive Failure Count: 0
Last Predictive Failure Event Seq Number: 0
PD Type: SAS

Raw Size: 68.365 GB [0x88bb6b0 Sectors]
Non Coerced Size: 67.865 GB [0x87bb6b0 Sectors]
Coerced Size: 67.054 GB [0x861c000 Sectors]
Sector Size:  0
Firmware state: Online, Spun Up
Device Firmware Level: B62C
Shield Counter: 0
Successful diagnostics completion on :  N/A
SAS Address(0): 0x5000c5002327a3e1
SAS Address(1): 0x0
Connected Port Number: 3(path0)
Inquiry Data: IBM-ESXSST973452SS      B62C3TA135670324B62C
IBM FRU/CRU: 43X0847
FDE Capable: Not Capable
FDE Enable: Disable
Secured: Unsecured
Locked: Unlocked
Needs EKM Attention: No
Foreign State: None
Device Speed: 6.0Gb/s
Link Speed: 3.0Gb/s
Media Type: Hard Disk Device
Drive Temperature :35C (95.00 F)
PI Eligibility:  No
Drive is formatted for PI information:  No
PI: No PI
Port-0 :
Port status: Active
Port's Linkspeed: 3.0Gb/s
Port-1 :
Port status: Active
Port's Linkspeed: Unknown
Drive has flagged a S.M.A.R.T alert : No



Enclosure Device ID: 252
Slot Number: 4
Enclosure position: N/A
Device Id: 5
WWN:
Sequence Number: 1
Media Error Count: 0
Other Error Count: 0
Predictive Failure Count: 0
Last Predictive Failure Event Seq Number: 0
PD Type: SAS

Raw Size: 136.731 GB [0x11176d60 Sectors]
Non Coerced Size: 136.231 GB [0x11076d60 Sectors]
Coerced Size: 135.972 GB [0x10ff2000 Sectors]
Sector Size:  0
Firmware state: Unconfigured(good), Spun Up
Device Firmware Level: B53E
Shield Counter: 0
Successful diagnostics completion on :  N/A
SAS Address(0): 0x5000c5001300ef29
SAS Address(1): 0x0
Connected Port Number: 4(path0)
Inquiry Data: IBM-ESXSST9146803SS     B53E3SD0JXWE0716B53E
IBM FRU/CRU: 42D0422
FDE Capable: Not Capable
FDE Enable: Disable
Secured: Unsecured
Locked: Unlocked
Needs EKM Attention: No
Foreign State: None
Device Speed: 6.0Gb/s
Link Speed: 3.0Gb/s
Media Type: Hard Disk Device
Drive Temperature :44C (111.20 F)
PI Eligibility:  No
Drive is formatted for PI information:  No
PI: No PI
Port-0 :
Port status: Active
Port's Linkspeed: 3.0Gb/s
Port-1 :
Port status: Active
Port's Linkspeed: Unknown
Drive has flagged a S.M.A.R.T alert : No



Enclosure Device ID: 252
Slot Number: 5
Enclosure position: N/A
Device Id: 2
WWN:
Sequence Number: 1
Media Error Count: 0
Other Error Count: 0
Predictive Failure Count: 0
Last Predictive Failure Event Seq Number: 0
PD Type: SAS

Raw Size: 136.731 GB [0x11176d60 Sectors]
Non Coerced Size: 136.231 GB [0x11076d60 Sectors]
Coerced Size: 135.972 GB [0x10ff2000 Sectors]
Sector Size:  0
Firmware state: Unconfigured(good), Spun Up
Device Firmware Level: B53E
Shield Counter: 0
Successful diagnostics completion on :  N/A
SAS Address(0): 0x5000c5001300e075
SAS Address(1): 0x0
Connected Port Number: 5(path0)
Inquiry Data: IBM-ESXSST9146803SS     B53E3SD0J38X0716B53E
IBM FRU/CRU: 42D0422
FDE Capable: Not Capable
FDE Enable: Disable
Secured: Unsecured
Locked: Unlocked
Needs EKM Attention: No
Foreign State: None
Device Speed: 6.0Gb/s
Link Speed: 3.0Gb/s
Media Type: Hard Disk Device
Drive Temperature :47C (116.60 F)
PI Eligibility:  No
Drive is formatted for PI information:  No
PI: No PI
Port-0 :
Port status: Active
Port's Linkspeed: 3.0Gb/s
Port-1 :
Port status: Active
Port's Linkspeed: Unknown
Drive has flagged a S.M.A.R.T alert : No



Enclosure Device ID: 252
Slot Number: 6
Enclosure position: N/A
Device Id: 4
WWN:
Sequence Number: 1
Media Error Count: 0
Other Error Count: 0
Predictive Failure Count: 0
Last Predictive Failure Event Seq Number: 0
PD Type: SAS

Raw Size: 136.731 GB [0x11176d60 Sectors]
Non Coerced Size: 136.231 GB [0x11076d60 Sectors]
Coerced Size: 135.972 GB [0x10ff2000 Sectors]
Sector Size:  0
Firmware state: Unconfigured(good), Spun Up
Device Firmware Level: B53E
Shield Counter: 0
Successful diagnostics completion on :  N/A
SAS Address(0): 0x5000c500130155b5
SAS Address(1): 0x0
Connected Port Number: 6(path0)
Inquiry Data: IBM-ESXSST9146803SS     B53E3SD0H8BW0716B53E
IBM FRU/CRU: 42D0422
FDE Capable: Not Capable
FDE Enable: Disable
Secured: Unsecured
Locked: Unlocked
Needs EKM Attention: No
Foreign State: None
Device Speed: 6.0Gb/s
Link Speed: 3.0Gb/s
Media Type: Hard Disk Device
Drive Temperature :44C (111.20 F)
PI Eligibility:  No
Drive is formatted for PI information:  No
PI: No PI
Port-0 :
Port status: Active
Port's Linkspeed: 3.0Gb/s
Port-1 :
Port status: Active
Port's Linkspeed: Unknown
Drive has flagged a S.M.A.R.T alert : No

 

Firmware on the test server

Firmware Type   Version String  Release Date
IMM     YUOOG2C-1.42    12/16/2013
UEFI    D6E161A-1.19    11/29/2013
DSA     DSYTC5C-9.42    11/05/2013

webbios:
 

 

 

Controller Details:

[root@flash ~]# /opt/MegaRAID/MegaCli/MegaCli64  -AdpAllInfo -aAll

Adapter #0

==============================================================================
                    Versions
                ================
Product Name    : IBM ServeRAID-MR10i SAS/SATA Controller
Serial No       : P140564309
FW Package Build: 11.0.1-0054

                    Mfg. Data
                ================
Mfg. Date       : 10/25/09
Rework Date     : 00/00/00
Revision No     :
Battery FRU     : N/A

                Image Versions in Flash:
                ================
FW Version         : 1.40.282-2321
BIOS Version       : 2.07.00
WebBIOS Version    : 2.2-22-e_14-Rel
Preboot CLI Version: 01.40-010:#%00008
Boot Block Version : 1.00.00.01-0012

                Pending Images in Flash
                ================
None

                PCI Info
                ================
Controller Id   : 0000
Vendor Id       : 1000
Device Id       : 0060
SubVendorId     : 1014
SubDeviceId     : 0364

Host Interface  : PCIE

Link Speed           : 0
Number of Frontend Port: 0
Device Interface  : PCIE

Number of Backend Port: 8
Port  :  Address
0        0000000000000000
1        0000000000000000
2        0000000000000000
3        5000c5002327a3e1
4        5000c5001300ef29
5        5000c5001300e075
6        5000c500130155b5
7        0000000000000000

                HW Configuration
                ================
SAS Address      : 500605b0017eed80
BBU              : Present
Alarm            : Present
NVRAM            : Present
Serial Debugger  : Present
Memory           : Present
Flash            : Present
Memory Size      : 256MB
TPM              : Absent
On board Expander: Absent
Upgrade Key      : Absent
Temperature sensor for ROC    : Absent
Temperature sensor for controller    : Absent


                Settings
                ================
Current Time                     : 1:9:36 3/3, 2014
Predictive Fail Poll Interval    : 300sec
Interrupt Throttle Active Count  : 16
Interrupt Throttle Completion    : 50us
Rebuild Rate                     : 30%
PR Rate                          : 30%
BGI Rate                         : 30%
Check Consistency Rate           : 30%
Reconstruction Rate              : 30%
Cache Flush Interval             : 4s
Max Drives to Spinup at One Time : 2
Delay Among Spinup Groups        : 12s
Physical Drive Coercion Mode     : 1GB
Cluster Mode                     : Disabled
Alarm                            : Enabled
Auto Rebuild                     : Enabled
Battery Warning                  : Enabled
Ecc Bucket Size                  : 15
Ecc Bucket Leak Rate             : 1440 Minutes
Restore HotSpare on Insertion    : Disabled
Expose Enclosure Devices         : Enabled
Maintain PD Fail History         : Enabled
Host Request Reordering          : Enabled
Auto Detect BackPlane Enabled    : SGPIO/i2c SEP
Load Balance Mode                : Auto
Use FDE Only                     : No
Security Key Assigned            : No
Security Key Failed              : No
Security Key Not Backedup        : No
Default LD PowerSave Policy      : Controller Defined
Maximum number of direct attached drives to spin up in 1 min : 0
Auto Enhanced Import             : No
Any Offline VD Cache Preserved   : No
Allow Boot with Preserved Cache  : No
Disable Online Controller Reset  : No
PFK in NVRAM                     : No
Use disk activity for locate     : No
POST delay                       : 90 seconds
BIOS Error Handling              : Stop On Errors
Current Boot Mode                 :Normal
                Capabilities
                ================
RAID Level Supported             : RAID0, RAID1, RAID5, RAID6, RAID00, RAID10, RAID50, RAID60, PRL 11, PRL 11 with spanning, SRL 3 supported, PRL11-RLQ0 DDF layout with no span, PRL11-RLQ0 DDF layout with span
Supported Drives                 : SAS, SATA

Allowed Mixing:

Mix in Enclosure Allowed

                Status
                ================
ECC Bucket Count                 : 0

                Limitations
                ================
Max Arms Per VD          : 32
Max Spans Per VD         : 8
Max Arrays               : 128
Max Number of VDs        : 64
Max Parallel Commands    : 1008
Max SGE Count            : 80
Max Data Transfer Size   : 8192 sectors
Max Strips PerIO         : 42
Max LD per array         : 16
Min Strip Size           : 8 KB
Max Strip Size           : 1.0 MB
Max Configurable CacheCade Size: 0 GB
Current Size of CacheCade      : 0 GB
Current Size of FW Cache       : 0 MB

                Device Present
                ================
Virtual Drives    : 1
  Degraded        : 0
  Offline         : 0
Physical Devices  : 5
  Disks           : 4
  Critical Disks  : 0
  Failed Disks    : 0

                Supported Adapter Operations
                ================
Rebuild Rate                    : Yes
CC Rate                         : Yes
BGI Rate                        : Yes
Reconstruct Rate                : Yes
Patrol Read Rate                : Yes
Alarm Control                   : Yes
Cluster Support                 : No
BBU                             : Yes
Spanning                        : Yes
Dedicated Hot Spare             : Yes
Revertible Hot Spares           : Yes
Foreign Config Import           : Yes
Self Diagnostic                 : Yes
Allow Mixed Redundancy on Array : No
Global Hot Spares               : Yes
Deny SCSI Passthrough           : No
Deny SMP Passthrough            : No
Deny STP Passthrough            : No
Support Security                : No
Snapshot Enabled                : No
Support the OCE without adding drives : No
Support PFK                     : No
Support PI                      : No
Support Boot Time PFK Change    : No
Disable Online PFK Change       : No
Support Shield State            : No
Block SSD Write Disk Cache Change: No

                Supported VD Operations
                ================
Read Policy          : Yes
Write Policy         : Yes
IO Policy            : Yes
Access Policy        : Yes
Disk Cache Policy    : Yes
Reconstruction       : Yes
Deny Locate          : No
Deny CC              : No
Allow Ctrl Encryption: No
Enable LDBBM         : No
Support Breakmirror  : No
Power Savings        : No

                Supported PD Operations
                ================
Force Online                            : Yes
Force Offline                           : Yes
Force Rebuild                           : Yes
Deny Force Failed                       : No
Deny Force Good/Bad                     : No
Deny Missing Replace                    : No
Deny Clear                              : No
Deny Locate                             : No
Support Temperature                     : No
Disable Copyback                        : No
Enable JBOD                             : No
Enable Copyback on SMART                : No
Enable Copyback to SSD on SMART Error   : Yes
Enable SSD Patrol Read                  : No
PR Correct Unconfigured Areas           : Yes
Enable Spin Down of UnConfigured Drives : No
Disable Spin Down of hot spares         : No
Spin Down time                          : 0
T10 Power State                         : No
                Error Counters
                ================
Memory Correctable Errors   : 0
Memory Uncorrectable Errors : 0

                Cluster Information
                ================
Cluster Permitted     : No
Cluster Active        : No

                Default Settings
                ================
Phy Polarity                     : 0
Phy PolaritySplit                : 0
Background Rate                  : 30
Strip Size                       : 128kB
Flush Time                       : 4 seconds
Write Policy                     : WB
Read Policy                      : None
Cache When BBU Bad               : Disabled
Cached IO                        : No
SMART Mode                       : Mode 6
Alarm Disable                    : Yes
Coercion Mode                    : 1GB
ZCR Config                       : Unknown
Dirty LED Shows Drive Activity   : No
BIOS Continue on Error           : 0
Spin Down Mode                   : None
Allowed Device Type              : SAS/SATA Mix
Allow Mix in Enclosure           : Yes
Allow HDD SAS/SATA Mix in VD     : No
Allow SSD SAS/SATA Mix in VD     : No
Allow HDD/SSD Mix in VD          : No
Allow SATA in Cluster            : No
Max Chained Enclosures           : 3
Disable Ctrl-R                   : Yes
Enable Web BIOS                  : Yes
Direct PD Mapping                : No
BIOS Enumerate VDs               : Yes
Restore Hot Spare on Insertion   : No
Expose Enclosure Devices         : Yes
Maintain PD Fail History         : Yes
Disable Puncturing               : Yes
Zero Based Enclosure Enumeration : No
PreBoot CLI Enabled              : Yes
LED Show Drive Activity          : No
Cluster Disable                  : Yes
SAS Disable                      : No
Auto Detect BackPlane Enable     : SGPIO/i2c SEP
Use FDE Only                     : No
Enable Led Header                : No
Delay during POST                : 0
EnableCrashDump                  : No
Disable Online Controller Reset  : No
EnableLDBBM                      : No
Un-Certified Hard Disk Drives    : Block
Treat Single span R1E as R10     : No
Max LD per array                 : 16
Power Saving option              : All power saving options are enabled
Default spin down time in minutes: 0
Enable JBOD                      : No
Time taken to detect CME         : 60s

 

Very strang is that I've in the logs the right ID's if a drive fails, then the system change it to the wrong, this will generates 2 errors for one drive, here:

Any Idea? I really have no budget for new controllers, I also invested to much for all the upgrades. I've no Idea why little updates make such a mess. I'll never ever upgrade my other ibm servers. Spend 3 days to get it working, with no luck.

 

BTW: warranty and purchased servcie packs expired 7 months ago, so I can't call ibm. I've an x3650 m3 here with another fault in warranty so the ibm tech has to come anyway but I think he will not help me on this

 

Thanks!

  • chris8135
    chris8135
    2 Posts

    Re: x3650 m2 mr10i after update with bomc backplane slots don't matches drive ids

    ‏2014-03-03T17:26:54Z  

    I've now tested 2 x3650 m2 servers with other controllers, m5016 and br10i, on this it works as expected so i'm now sure the problem is not the server firmware

     

    m5016:

     

    mr10i

     

    I've also tested the mr10i in x3650 m3 and x3650 m4, on this systems it works:

     

     

    So, does somebody know how to downgrade? I don't want to use the old controllers in my new x3650 servers

    I've tried to flash dell perc firmware with no luck

     

     

    Updated on 2014-03-03T17:28:39Z at 2014-03-03T17:28:39Z by chris8135
  • Novikov_Alexander
    Novikov_Alexander
    6877 Posts

    Re: x3650 m2 mr10i after update with bomc backplane slots don't matches drive ids

    ‏2014-03-21T06:31:58Z  
    • chris8135
    • ‏2014-03-03T17:26:54Z

    I've now tested 2 x3650 m2 servers with other controllers, m5016 and br10i, on this it works as expected so i'm now sure the problem is not the server firmware

     

    m5016:

     

    mr10i

     

    I've also tested the mr10i in x3650 m3 and x3650 m4, on this systems it works:

     

     

    So, does somebody know how to downgrade? I don't want to use the old controllers in my new x3650 servers

    I've tried to flash dell perc firmware with no luck

     

     

    Dear  Christian,

    could you post DSA log of your server.

    Regards,
    Alexander Novikov
    Russia, Moscow