Topic
2 replies Latest Post - ‏2014-03-21T06:31:58Z by Novikov_Alexander
chris8135
chris8135
2 Posts
ACCEPTED ANSWER

Pinned topic x3650 m2 mr10i after update with bomc backplane slots don't matches drive ids

‏2014-03-03T00:36:51Z |

Hello,

I've 5 x3650 m2 servers here, pulled from dc after working 3y, I want to use it now for other tasks. I've ordered new drives, new cpus, upgrades and new ram to put it to a new level and upgrade all to strong servers with the same power like x3650 m3 which I also owns. Worked all fine until I did an upgrade of all with update cd created with bomc. Unfortunately I never checked the raid controller, until I tried to add an sas expansion card to add 4 more drives to one of it then I noticed that the backplane id's don't matches the physical drive id's. First I thought the sas expander from the upgrade kit is the faulty part, so I've removed it, same error in webbios. I checked the other 4, on all servers the same :-(

I used the latest serveraid formware from ibm site, I tried downgrades, all possible solutions failed. So I decided to use the latest lsi firmware form lso site:

[root@flash ~]# /opt/MegaRAID/MegaCli/MegaCli64  -adpfwflash -f mr1078fw.rom -a0

Adapter 0: IBM ServeRAID-MR10i SAS/SATA Controller
Vendor ID: 0x1000, Device ID: 0x0060

Package version on the controller: 11.0.1-0042
Package version of the image file: 11.0.1-0054
Download Completed.
Flashing image to adapter...
Adapter 0: Flash Completed.
Please reboot the system for the changes to take effect

Exit Code: 0x00

Still wrong ID's. Any Idea whats wrong? I found another user here with the same problem: https://www.ibm.com/developerworks/community/forums/html/topic?id=77777777-0000-0000-0000-000014893499#77777777-0000-0000-0000-000014898797

I can't use the servers this way, if a drives fails and somebody other changes it and pull the wrong than the array will be distroyed...

Test Server (8 HDD backplane, no expander) has the following test cfg:

0 - 146GB
1 - 146GB
2 - 146GB
7 - 73GB

This is what I get:

[root@flash ~]# /opt/MegaRAID/MegaCli/MegaCli64  -PDList -a0

Adapter #0

Enclosure Device ID: 252
Slot Number: 3
Drive's position: DiskGroup: 0, Span: 0, Arm: 0
Enclosure position: N/A
Device Id: 9
WWN:
Sequence Number: 2
Media Error Count: 0
Other Error Count: 0
Predictive Failure Count: 0
Last Predictive Failure Event Seq Number: 0
PD Type: SAS

Raw Size: 68.365 GB [0x88bb6b0 Sectors]
Non Coerced Size: 67.865 GB [0x87bb6b0 Sectors]
Coerced Size: 67.054 GB [0x861c000 Sectors]
Sector Size:  0
Firmware state: Online, Spun Up
Device Firmware Level: B62C
Shield Counter: 0
Successful diagnostics completion on :  N/A
SAS Address(0): 0x5000c5002327a3e1
SAS Address(1): 0x0
Connected Port Number: 3(path0)
Inquiry Data: IBM-ESXSST973452SS      B62C3TA135670324B62C
IBM FRU/CRU: 43X0847
FDE Capable: Not Capable
FDE Enable: Disable
Secured: Unsecured
Locked: Unlocked
Needs EKM Attention: No
Foreign State: None
Device Speed: 6.0Gb/s
Link Speed: 3.0Gb/s
Media Type: Hard Disk Device
Drive Temperature :35C (95.00 F)
PI Eligibility:  No
Drive is formatted for PI information:  No
PI: No PI
Port-0 :
Port status: Active
Port's Linkspeed: 3.0Gb/s
Port-1 :
Port status: Active
Port's Linkspeed: Unknown
Drive has flagged a S.M.A.R.T alert : No



Enclosure Device ID: 252
Slot Number: 4
Enclosure position: N/A
Device Id: 5
WWN:
Sequence Number: 1
Media Error Count: 0
Other Error Count: 0
Predictive Failure Count: 0
Last Predictive Failure Event Seq Number: 0
PD Type: SAS

Raw Size: 136.731 GB [0x11176d60 Sectors]
Non Coerced Size: 136.231 GB [0x11076d60 Sectors]
Coerced Size: 135.972 GB [0x10ff2000 Sectors]
Sector Size:  0
Firmware state: Unconfigured(good), Spun Up
Device Firmware Level: B53E
Shield Counter: 0
Successful diagnostics completion on :  N/A
SAS Address(0): 0x5000c5001300ef29
SAS Address(1): 0x0
Connected Port Number: 4(path0)
Inquiry Data: IBM-ESXSST9146803SS     B53E3SD0JXWE0716B53E
IBM FRU/CRU: 42D0422
FDE Capable: Not Capable
FDE Enable: Disable
Secured: Unsecured
Locked: Unlocked
Needs EKM Attention: No
Foreign State: None
Device Speed: 6.0Gb/s
Link Speed: 3.0Gb/s
Media Type: Hard Disk Device
Drive Temperature :44C (111.20 F)
PI Eligibility:  No
Drive is formatted for PI information:  No
PI: No PI
Port-0 :
Port status: Active
Port's Linkspeed: 3.0Gb/s
Port-1 :
Port status: Active
Port's Linkspeed: Unknown
Drive has flagged a S.M.A.R.T alert : No



Enclosure Device ID: 252
Slot Number: 5
Enclosure position: N/A
Device Id: 2
WWN:
Sequence Number: 1
Media Error Count: 0
Other Error Count: 0
Predictive Failure Count: 0
Last Predictive Failure Event Seq Number: 0
PD Type: SAS

Raw Size: 136.731 GB [0x11176d60 Sectors]
Non Coerced Size: 136.231 GB [0x11076d60 Sectors]
Coerced Size: 135.972 GB [0x10ff2000 Sectors]
Sector Size:  0
Firmware state: Unconfigured(good), Spun Up
Device Firmware Level: B53E
Shield Counter: 0
Successful diagnostics completion on :  N/A
SAS Address(0): 0x5000c5001300e075
SAS Address(1): 0x0
Connected Port Number: 5(path0)
Inquiry Data: IBM-ESXSST9146803SS     B53E3SD0J38X0716B53E
IBM FRU/CRU: 42D0422
FDE Capable: Not Capable
FDE Enable: Disable
Secured: Unsecured
Locked: Unlocked
Needs EKM Attention: No
Foreign State: None
Device Speed: 6.0Gb/s
Link Speed: 3.0Gb/s
Media Type: Hard Disk Device
Drive Temperature :47C (116.60 F)
PI Eligibility:  No
Drive is formatted for PI information:  No
PI: No PI
Port-0 :
Port status: Active
Port's Linkspeed: 3.0Gb/s
Port-1 :
Port status: Active
Port's Linkspeed: Unknown
Drive has flagged a S.M.A.R.T alert : No



Enclosure Device ID: 252
Slot Number: 6
Enclosure position: N/A
Device Id: 4
WWN:
Sequence Number: 1
Media Error Count: 0
Other Error Count: 0
Predictive Failure Count: 0
Last Predictive Failure Event Seq Number: 0
PD Type: SAS

Raw Size: 136.731 GB [0x11176d60 Sectors]
Non Coerced Size: 136.231 GB [0x11076d60 Sectors]
Coerced Size: 135.972 GB [0x10ff2000 Sectors]
Sector Size:  0
Firmware state: Unconfigured(good), Spun Up
Device Firmware Level: B53E
Shield Counter: 0
Successful diagnostics completion on :  N/A
SAS Address(0): 0x5000c500130155b5
SAS Address(1): 0x0
Connected Port Number: 6(path0)
Inquiry Data: IBM-ESXSST9146803SS     B53E3SD0H8BW0716B53E
IBM FRU/CRU: 42D0422
FDE Capable: Not Capable
FDE Enable: Disable
Secured: Unsecured
Locked: Unlocked
Needs EKM Attention: No
Foreign State: None
Device Speed: 6.0Gb/s
Link Speed: 3.0Gb/s
Media Type: Hard Disk Device
Drive Temperature :44C (111.20 F)
PI Eligibility:  No
Drive is formatted for PI information:  No
PI: No PI
Port-0 :
Port status: Active
Port's Linkspeed: 3.0Gb/s
Port-1 :
Port status: Active
Port's Linkspeed: Unknown
Drive has flagged a S.M.A.R.T alert : No

 

Firmware on the test server

Firmware Type   Version String  Release Date
IMM     YUOOG2C-1.42    12/16/2013
UEFI    D6E161A-1.19    11/29/2013
DSA     DSYTC5C-9.42    11/05/2013

webbios:
 

 

 

Controller Details:

[root@flash ~]# /opt/MegaRAID/MegaCli/MegaCli64  -AdpAllInfo -aAll

Adapter #0

==============================================================================
                    Versions
                ================
Product Name    : IBM ServeRAID-MR10i SAS/SATA Controller
Serial No       : P140564309
FW Package Build: 11.0.1-0054

                    Mfg. Data
                ================
Mfg. Date       : 10/25/09
Rework Date     : 00/00/00
Revision No     :
Battery FRU     : N/A

                Image Versions in Flash:
                ================
FW Version         : 1.40.282-2321
BIOS Version       : 2.07.00
WebBIOS Version    : 2.2-22-e_14-Rel
Preboot CLI Version: 01.40-010:#%00008
Boot Block Version : 1.00.00.01-0012

                Pending Images in Flash
                ================
None

                PCI Info
                ================
Controller Id   : 0000
Vendor Id       : 1000
Device Id       : 0060
SubVendorId     : 1014
SubDeviceId     : 0364

Host Interface  : PCIE

Link Speed           : 0
Number of Frontend Port: 0
Device Interface  : PCIE

Number of Backend Port: 8
Port  :  Address
0        0000000000000000
1        0000000000000000
2        0000000000000000
3        5000c5002327a3e1
4        5000c5001300ef29
5        5000c5001300e075
6        5000c500130155b5
7        0000000000000000

                HW Configuration
                ================
SAS Address      : 500605b0017eed80
BBU              : Present
Alarm            : Present
NVRAM            : Present
Serial Debugger  : Present
Memory           : Present
Flash            : Present
Memory Size      : 256MB
TPM              : Absent
On board Expander: Absent
Upgrade Key      : Absent
Temperature sensor for ROC    : Absent
Temperature sensor for controller    : Absent


                Settings
                ================
Current Time                     : 1:9:36 3/3, 2014
Predictive Fail Poll Interval    : 300sec
Interrupt Throttle Active Count  : 16
Interrupt Throttle Completion    : 50us
Rebuild Rate                     : 30%
PR Rate                          : 30%
BGI Rate                         : 30%
Check Consistency Rate           : 30%
Reconstruction Rate              : 30%
Cache Flush Interval             : 4s
Max Drives to Spinup at One Time : 2
Delay Among Spinup Groups        : 12s
Physical Drive Coercion Mode     : 1GB
Cluster Mode                     : Disabled
Alarm                            : Enabled
Auto Rebuild                     : Enabled
Battery Warning                  : Enabled
Ecc Bucket Size                  : 15
Ecc Bucket Leak Rate             : 1440 Minutes
Restore HotSpare on Insertion    : Disabled
Expose Enclosure Devices         : Enabled
Maintain PD Fail History         : Enabled
Host Request Reordering          : Enabled
Auto Detect BackPlane Enabled    : SGPIO/i2c SEP
Load Balance Mode                : Auto
Use FDE Only                     : No
Security Key Assigned            : No
Security Key Failed              : No
Security Key Not Backedup        : No
Default LD PowerSave Policy      : Controller Defined
Maximum number of direct attached drives to spin up in 1 min : 0
Auto Enhanced Import             : No
Any Offline VD Cache Preserved   : No
Allow Boot with Preserved Cache  : No
Disable Online Controller Reset  : No
PFK in NVRAM                     : No
Use disk activity for locate     : No
POST delay                       : 90 seconds
BIOS Error Handling              : Stop On Errors
Current Boot Mode                 :Normal
                Capabilities
                ================
RAID Level Supported             : RAID0, RAID1, RAID5, RAID6, RAID00, RAID10, RAID50, RAID60, PRL 11, PRL 11 with spanning, SRL 3 supported, PRL11-RLQ0 DDF layout with no span, PRL11-RLQ0 DDF layout with span
Supported Drives                 : SAS, SATA

Allowed Mixing:

Mix in Enclosure Allowed

                Status
                ================
ECC Bucket Count                 : 0

                Limitations
                ================
Max Arms Per VD          : 32
Max Spans Per VD         : 8
Max Arrays               : 128
Max Number of VDs        : 64
Max Parallel Commands    : 1008
Max SGE Count            : 80
Max Data Transfer Size   : 8192 sectors
Max Strips PerIO         : 42
Max LD per array         : 16
Min Strip Size           : 8 KB
Max Strip Size           : 1.0 MB
Max Configurable CacheCade Size: 0 GB
Current Size of CacheCade      : 0 GB
Current Size of FW Cache       : 0 MB

                Device Present
                ================
Virtual Drives    : 1
  Degraded        : 0
  Offline         : 0
Physical Devices  : 5
  Disks           : 4
  Critical Disks  : 0
  Failed Disks    : 0

                Supported Adapter Operations
                ================
Rebuild Rate                    : Yes
CC Rate                         : Yes
BGI Rate                        : Yes
Reconstruct Rate                : Yes
Patrol Read Rate                : Yes
Alarm Control                   : Yes
Cluster Support                 : No
BBU                             : Yes
Spanning                        : Yes
Dedicated Hot Spare             : Yes
Revertible Hot Spares           : Yes
Foreign Config Import           : Yes
Self Diagnostic                 : Yes
Allow Mixed Redundancy on Array : No
Global Hot Spares               : Yes
Deny SCSI Passthrough           : No
Deny SMP Passthrough            : No
Deny STP Passthrough            : No
Support Security                : No
Snapshot Enabled                : No
Support the OCE without adding drives : No
Support PFK                     : No
Support PI                      : No
Support Boot Time PFK Change    : No
Disable Online PFK Change       : No
Support Shield State            : No
Block SSD Write Disk Cache Change: No

                Supported VD Operations
                ================
Read Policy          : Yes
Write Policy         : Yes
IO Policy            : Yes
Access Policy        : Yes
Disk Cache Policy    : Yes
Reconstruction       : Yes
Deny Locate          : No
Deny CC              : No
Allow Ctrl Encryption: No
Enable LDBBM         : No
Support Breakmirror  : No
Power Savings        : No

                Supported PD Operations
                ================
Force Online                            : Yes
Force Offline                           : Yes
Force Rebuild                           : Yes
Deny Force Failed                       : No
Deny Force Good/Bad                     : No
Deny Missing Replace                    : No
Deny Clear                              : No
Deny Locate                             : No
Support Temperature                     : No
Disable Copyback                        : No
Enable JBOD                             : No
Enable Copyback on SMART                : No
Enable Copyback to SSD on SMART Error   : Yes
Enable SSD Patrol Read                  : No
PR Correct Unconfigured Areas           : Yes
Enable Spin Down of UnConfigured Drives : No
Disable Spin Down of hot spares         : No
Spin Down time                          : 0
T10 Power State                         : No
                Error Counters
                ================
Memory Correctable Errors   : 0
Memory Uncorrectable Errors : 0

                Cluster Information
                ================
Cluster Permitted     : No
Cluster Active        : No

                Default Settings
                ================
Phy Polarity                     : 0
Phy PolaritySplit                : 0
Background Rate                  : 30
Strip Size                       : 128kB
Flush Time                       : 4 seconds
Write Policy                     : WB
Read Policy                      : None
Cache When BBU Bad               : Disabled
Cached IO                        : No
SMART Mode                       : Mode 6
Alarm Disable                    : Yes
Coercion Mode                    : 1GB
ZCR Config                       : Unknown
Dirty LED Shows Drive Activity   : No
BIOS Continue on Error           : 0
Spin Down Mode                   : None
Allowed Device Type              : SAS/SATA Mix
Allow Mix in Enclosure           : Yes
Allow HDD SAS/SATA Mix in VD     : No
Allow SSD SAS/SATA Mix in VD     : No
Allow HDD/SSD Mix in VD          : No
Allow SATA in Cluster            : No
Max Chained Enclosures           : 3
Disable Ctrl-R                   : Yes
Enable Web BIOS                  : Yes
Direct PD Mapping                : No
BIOS Enumerate VDs               : Yes
Restore Hot Spare on Insertion   : No
Expose Enclosure Devices         : Yes
Maintain PD Fail History         : Yes
Disable Puncturing               : Yes
Zero Based Enclosure Enumeration : No
PreBoot CLI Enabled              : Yes
LED Show Drive Activity          : No
Cluster Disable                  : Yes
SAS Disable                      : No
Auto Detect BackPlane Enable     : SGPIO/i2c SEP
Use FDE Only                     : No
Enable Led Header                : No
Delay during POST                : 0
EnableCrashDump                  : No
Disable Online Controller Reset  : No
EnableLDBBM                      : No
Un-Certified Hard Disk Drives    : Block
Treat Single span R1E as R10     : No
Max LD per array                 : 16
Power Saving option              : All power saving options are enabled
Default spin down time in minutes: 0
Enable JBOD                      : No
Time taken to detect CME         : 60s

 

Very strang is that I've in the logs the right ID's if a drive fails, then the system change it to the wrong, this will generates 2 errors for one drive, here:

Any Idea? I really have no budget for new controllers, I also invested to much for all the upgrades. I've no Idea why little updates make such a mess. I'll never ever upgrade my other ibm servers. Spend 3 days to get it working, with no luck.

 

BTW: warranty and purchased servcie packs expired 7 months ago, so I can't call ibm. I've an x3650 m3 here with another fault in warranty so the ibm tech has to come anyway but I think he will not help me on this

 

Thanks!

  • chris8135
    chris8135
    2 Posts
    ACCEPTED ANSWER

    Re: x3650 m2 mr10i after update with bomc backplane slots don't matches drive ids

    ‏2014-03-03T17:26:54Z  in response to chris8135

    I've now tested 2 x3650 m2 servers with other controllers, m5016 and br10i, on this it works as expected so i'm now sure the problem is not the server firmware

     

    m5016:

     

    mr10i

     

    I've also tested the mr10i in x3650 m3 and x3650 m4, on this systems it works:

     

     

    So, does somebody know how to downgrade? I don't want to use the old controllers in my new x3650 servers

    I've tried to flash dell perc firmware with no luck

     

     

    Updated on 2014-03-03T17:28:39Z at 2014-03-03T17:28:39Z by chris8135
    • Novikov_Alexander
      Novikov_Alexander
      6863 Posts
      ACCEPTED ANSWER

      Re: x3650 m2 mr10i after update with bomc backplane slots don't matches drive ids

      ‏2014-03-21T06:31:58Z  in response to chris8135

      Dear  Christian,

      could you post DSA log of your server.

      Regards,
      Alexander Novikov
      Russia, Moscow