Topic
  • 7 replies
  • Latest Post - ‏2013-01-07T19:42:58Z by Novikov_Alexander
SystemAdmin
SystemAdmin
4779 Posts

Pinned topic DS3300 Firmware upgrade failed...

‏2013-01-03T23:05:22Z |
Hi Guys,

We have a spare DS3300.

I used DS SM V10 to upgrade the Firmware from V6 to the latest V7.

I first transferred the firmware on to the device. It was successful.

After I activated the virson, there was an error message saying upgrade failed, lost connection to the device.

The SAN was left running over night.

I still cannot connect to it. controller A is pingable, B is not.

There are yellow lights on the controllers. B's ethernet port has no light.

If I reboot the SAN, the controllers will respond to ping initially. Then B dies.

Any suggestions will be much appreciated.

Thanks.
Updated on 2013-01-07T19:42:58Z at 2013-01-07T19:42:58Z by Novikov_Alexander
  • SystemAdmin
    SystemAdmin
    4779 Posts

    Re: DS3300 Firmware upgrade failed...

    ‏2013-01-04T00:09:59Z  
    I pull some info out of the recovery folder on my laptop (C:\Program Files (x86)\IBM_DS\client\data\recovery).
    The old configure has issues. Thus I pulled out the hard drive.
    Before I started doing the upgrade, there were no logical drivers configured, no issue, system status was all green.
    There were 3 non assigned hard drives in the SAN.

    TimeStamp 1 3/01/2013 16:19 1357190372952
    Header 600A0B800036E71E0000000050E51383 Unnamed 1
    CtlFirmware SF73500075 06.70.24.00
    CtlFirmware SF72900326 06.70.24.00
    VolumeGroup 600A0B8000370E59000003C6482791B9 1 5 TRUE 5000CCA20EDE6C6F
    Volume 600A0B800036E71E0000032648272DDF 2 600A0B8000370E59000003C6482791B9 512 86461382656 594193152 131072 2 5
    Volume 600A0B8000370E59000003C548272EA6 1 600A0B8000370E59000003C6482791B9 512 912680550400 0 131072 2 5
    Drive 5000CCA20EDE6C6F 85,3 KRVN97ZBJ4XSDH 600A0B8000370E59000003C6482791B9 1 FALSE
    Drive 85,11 600A0B8000370E59000003C6482791B9 6 FALSE

    TimeStamp 1 3/01/2013 16:20 1357190441045
    Drive 5000CCA20EDE6C6F 85,3 KRVN97ZBJ4XSDH 600A0B8000370E59000003C6482791B9 6 FALSE

    TimeStamp 1 3/01/2013 16:22 1357190561574
    VolumeGroup 600A0B8000370E59000003C6482791B9 Missing
    Volume 600A0B8000370E59000003C548272EA6 Missing
    Volume 600A0B800036E71E0000032648272DDF Missing

    TimeStamp 1 3/01/2013 16:23 1357190618741
    VolumeGroup 600A0B8000370E59000003C6482791B9 1 5 TRUE 5000CCA20EDE6C6F
    Volume 600A0B800036E71E0000032648272DDF 2 600A0B8000370E59000003C6482791B9 512 86461382656 594193152 131072 2 5
    Volume 600A0B8000370E59000003C548272EA6 1 600A0B8000370E59000003C6482791B9 512 912680550400 0 131072 2 5
    Drive 5000CCA20EDE6C6F 85,3 KRVN97ZBJ4XSDH 600A0B8000370E59000003C6482791B9 1 FALSE
    Drive 85,11 600A0B8000370E59000003C6482791B9 6 FALSE

    TimeStamp 1 3/01/2013 16:30 1357191042580
    Drive 5000CCA20EDE6C6F 85,3 KRVN97ZBJ4XSDH 600A0B8000370E59000003C6482791B9 6 FALSE

    TimeStamp 1 3/01/2013 16:30 1357191059323
    VolumeGroup 600A0B8000370E59000003C6482791B9 1 5 TRUE 5000CCA20EDE9925 5000CCA20EDE6C6F
    Drive 5000CCA20EDE9925 85,2 KRVN97ZBJ59NMH 600A0B8000370E59000003C6482791B9 1 FALSE

    TimeStamp 1 3/01/2013 16:34 1357191282030
    VolumeGroup 600A0B8000370E59000003C6482791B9 Missing
    Volume 600A0B8000370E59000003C548272EA6 Missing
    Volume 600A0B800036E71E0000032648272DDF Missing

    TimeStamp 1 3/01/2013 16:46 1357191965504
    CtlFirmware SF73500075 07.35.71.00
  • SystemAdmin
    SystemAdmin
    4779 Posts

    Re: DS3300 Firmware upgrade failed...

    ‏2013-01-04T00:13:29Z  
    I pull some info out of the recovery folder on my laptop (C:\Program Files (x86)\IBM_DS\client\data\recovery).
    The old configure has issues. Thus I pulled out the hard drive.
    Before I started doing the upgrade, there were no logical drivers configured, no issue, system status was all green.
    There were 3 non assigned hard drives in the SAN.

    TimeStamp 1 3/01/2013 16:19 1357190372952
    Header 600A0B800036E71E0000000050E51383 Unnamed 1
    CtlFirmware SF73500075 06.70.24.00
    CtlFirmware SF72900326 06.70.24.00
    VolumeGroup 600A0B8000370E59000003C6482791B9 1 5 TRUE 5000CCA20EDE6C6F
    Volume 600A0B800036E71E0000032648272DDF 2 600A0B8000370E59000003C6482791B9 512 86461382656 594193152 131072 2 5
    Volume 600A0B8000370E59000003C548272EA6 1 600A0B8000370E59000003C6482791B9 512 912680550400 0 131072 2 5
    Drive 5000CCA20EDE6C6F 85,3 KRVN97ZBJ4XSDH 600A0B8000370E59000003C6482791B9 1 FALSE
    Drive 85,11 600A0B8000370E59000003C6482791B9 6 FALSE

    TimeStamp 1 3/01/2013 16:20 1357190441045
    Drive 5000CCA20EDE6C6F 85,3 KRVN97ZBJ4XSDH 600A0B8000370E59000003C6482791B9 6 FALSE

    TimeStamp 1 3/01/2013 16:22 1357190561574
    VolumeGroup 600A0B8000370E59000003C6482791B9 Missing
    Volume 600A0B8000370E59000003C548272EA6 Missing
    Volume 600A0B800036E71E0000032648272DDF Missing

    TimeStamp 1 3/01/2013 16:23 1357190618741
    VolumeGroup 600A0B8000370E59000003C6482791B9 1 5 TRUE 5000CCA20EDE6C6F
    Volume 600A0B800036E71E0000032648272DDF 2 600A0B8000370E59000003C6482791B9 512 86461382656 594193152 131072 2 5
    Volume 600A0B8000370E59000003C548272EA6 1 600A0B8000370E59000003C6482791B9 512 912680550400 0 131072 2 5
    Drive 5000CCA20EDE6C6F 85,3 KRVN97ZBJ4XSDH 600A0B8000370E59000003C6482791B9 1 FALSE
    Drive 85,11 600A0B8000370E59000003C6482791B9 6 FALSE

    TimeStamp 1 3/01/2013 16:30 1357191042580
    Drive 5000CCA20EDE6C6F 85,3 KRVN97ZBJ4XSDH 600A0B8000370E59000003C6482791B9 6 FALSE

    TimeStamp 1 3/01/2013 16:30 1357191059323
    VolumeGroup 600A0B8000370E59000003C6482791B9 1 5 TRUE 5000CCA20EDE9925 5000CCA20EDE6C6F
    Drive 5000CCA20EDE9925 85,2 KRVN97ZBJ59NMH 600A0B8000370E59000003C6482791B9 1 FALSE

    TimeStamp 1 3/01/2013 16:34 1357191282030
    VolumeGroup 600A0B8000370E59000003C6482791B9 Missing
    Volume 600A0B8000370E59000003C548272EA6 Missing
    Volume 600A0B800036E71E0000032648272DDF Missing

    TimeStamp 1 3/01/2013 16:46 1357191965504
    CtlFirmware SF73500075 07.35.71.00
    Here is the upgrade report.

    03/01/2013 4:35:08 PM Unnamed pre-upgrade tests pre-upgrade tests start
    Pre-upgrade tests started

    03/01/2013 4:35:12 PM Unnamed pre-upgrade tests check controller state
    Test passed - controller A

    03/01/2013 4:35:12 PM Unnamed pre-upgrade tests check controller state
    Test passed - controller B

    03/01/2013 4:35:13 PM Unnamed pre-upgrade tests check spm database
    Test passed - controller A

    03/01/2013 4:35:14 PM Unnamed pre-upgrade tests check spm database
    Test passed - controller B

    03/01/2013 4:35:14 PM Unnamed pre-upgrade tests Drives Unavailable
    Test passed

    03/01/2013 4:35:14 PM Unnamed pre-upgrade tests check drives
    Test passed

    03/01/2013 4:35:14 PM Unnamed pre-upgrade tests check arrays and disk pools
    Test passed

    03/01/2013 4:35:14 PM Unnamed pre-upgrade tests check hot spares
    Test passed

    03/01/2013 4:35:14 PM Unnamed pre-upgrade tests check current operations
    Test passed

    03/01/2013 4:35:14 PM Unnamed pre-upgrade tests check logical drives
    Test passed

    03/01/2013 4:35:20 PM Unnamed pre-upgrade tests check internal data validity
    Test passed

    03/01/2013 4:35:23 PM Unnamed pre-upgrade tests check event log
    The event log has not been cleared within the last 24 hours.

    03/01/2013 4:35:26 PM Unnamed pre-upgrade tests check event log
    Test passed

    03/01/2013 4:35:26 PM Unnamed pre-upgrade tests pre-upgrade tests complete 0h 0m 18s
    Pre-upgrade tests - all passed

    03/01/2013 4:35:27 PM Unnamed pre-upgrade tests pre-upgrade tests start
    Pre-upgrade tests started

    03/01/2013 4:35:28 PM Unnamed pre-upgrade tests check controller state
    Test passed - controller A

    03/01/2013 4:35:28 PM Unnamed pre-upgrade tests check controller state
    Test passed - controller B

    03/01/2013 4:35:28 PM Unnamed pre-upgrade tests check spm database
    Test passed - controller A

    03/01/2013 4:35:28 PM Unnamed pre-upgrade tests check spm database
    Test passed - controller B

    03/01/2013 4:35:28 PM Unnamed pre-upgrade tests Drives Unavailable
    Test passed

    03/01/2013 4:35:28 PM Unnamed pre-upgrade tests check drives
    Test passed

    03/01/2013 4:35:28 PM Unnamed pre-upgrade tests check arrays and disk pools
    Test passed

    03/01/2013 4:35:28 PM Unnamed pre-upgrade tests check hot spares
    Test passed

    03/01/2013 4:35:28 PM Unnamed pre-upgrade tests check current operations
    Test passed

    03/01/2013 4:35:28 PM Unnamed pre-upgrade tests check logical drives
    Test passed

    03/01/2013 4:35:34 PM Unnamed pre-upgrade tests check internal data validity
    Test passed

    03/01/2013 4:35:38 PM Unnamed pre-upgrade tests check event log
    The event log has not been cleared within the last 24 hours.

    03/01/2013 4:35:41 PM Unnamed pre-upgrade tests check event log
    Test passed

    03/01/2013 4:35:41 PM Unnamed pre-upgrade tests pre-upgrade tests complete 0h 0m 14s
    Pre-upgrade tests - all passed

    03/01/2013 4:35:55 PM Unnamed pre-upgrade tests pre-upgrade tests start
    Pre-upgrade tests started

    03/01/2013 4:35:56 PM Unnamed pre-upgrade tests Drives Unavailable
    Test passed

    03/01/2013 4:35:57 PM Unnamed pre-upgrade tests check dac store
    Test passed

    03/01/2013 4:35:57 PM Unnamed pre-upgrade tests pre-upgrade tests complete 0h 0m 1s
    Pre-upgrade tests - all passed

    03/01/2013 4:36:32 PM Unnamed SUPPORT_SERVICES start()
    Thread has been started

    03/01/2013 4:36:32 PM Unnamed SUPPORT_SERVICES DownloadOnly
    starting download only

    03/01/2013 4:36:32 PM Unnamed SUPPORT_SERVICES DownloadOnly
    nvsram file is NOT null, try to download both

    03/01/2013 4:36:32 PM Unnamed SUPPORT_SERVICES DownloadOnly
    download is starting

    03/01/2013 4:41:38 PM Unnamed SUPPORT_SERVICES DownloadOnly 0h 5m 5s
    download completed successfully

    03/01/2013 4:41:45 PM Unnamed pre-upgrade tests pre-upgrade tests start
    Pre-upgrade tests started

    03/01/2013 4:41:48 PM Unnamed pre-upgrade tests check controller state
    Test passed - controller A

    03/01/2013 4:41:48 PM Unnamed pre-upgrade tests check controller state
    Test passed - controller B

    03/01/2013 4:41:51 PM Unnamed pre-upgrade tests check spm database
    Test passed - controller A

    03/01/2013 4:41:53 PM Unnamed pre-upgrade tests check spm database
    Test passed - controller B

    03/01/2013 4:41:53 PM Unnamed pre-upgrade tests Drives Unavailable
    Test passed

    03/01/2013 4:41:53 PM Unnamed pre-upgrade tests check drives
    Test passed

    03/01/2013 4:41:53 PM Unnamed pre-upgrade tests check arrays and disk pools
    Test passed

    03/01/2013 4:41:53 PM Unnamed pre-upgrade tests check hot spares
    Test passed

    03/01/2013 4:41:53 PM Unnamed pre-upgrade tests check current operations
    Test passed

    03/01/2013 4:41:53 PM Unnamed pre-upgrade tests check logical drives
    Test passed

    03/01/2013 4:41:57 PM Unnamed pre-upgrade tests check internal data validity
    Test passed

    03/01/2013 4:42:00 PM Unnamed pre-upgrade tests check event log
    The event log has not been cleared within the last 24 hours.

    03/01/2013 4:42:03 PM Unnamed pre-upgrade tests check event log
    Test passed

    03/01/2013 4:42:03 PM Unnamed pre-upgrade tests pre-upgrade tests complete 0h 0m 18s
    Pre-upgrade tests - all passed

    03/01/2013 4:42:03 PM Unnamed pre-upgrade tests pre-upgrade tests start
    Pre-upgrade tests started

    03/01/2013 4:42:03 PM Unnamed pre-upgrade tests pre-upgrade tests start
    Pre-upgrade tests started

    03/01/2013 4:42:06 PM Unnamed pre-upgrade tests Drives Unavailable
    Test passed

    03/01/2013 4:42:06 PM Unnamed pre-upgrade tests pre-upgrade tests complete 0h 0m 2s
    Pre-upgrade tests - all passed

    03/01/2013 4:42:07 PM Unnamed pre-upgrade tests check controller state
    Test passed - controller A

    03/01/2013 4:42:07 PM Unnamed pre-upgrade tests check controller state
    Test passed - controller B

    03/01/2013 4:42:07 PM Unnamed pre-upgrade tests check spm database
    Test passed - controller A

    03/01/2013 4:42:09 PM Unnamed pre-upgrade tests check spm database
    Test passed - controller B

    03/01/2013 4:42:09 PM Unnamed pre-upgrade tests Drives Unavailable
    Test passed

    03/01/2013 4:42:09 PM Unnamed pre-upgrade tests check drives
    Test passed

    03/01/2013 4:42:09 PM Unnamed pre-upgrade tests check arrays and disk pools
    Test passed

    03/01/2013 4:42:09 PM Unnamed pre-upgrade tests check hot spares
    Test passed

    03/01/2013 4:42:09 PM Unnamed pre-upgrade tests check current operations
    Test passed

    03/01/2013 4:42:09 PM Unnamed pre-upgrade tests check logical drives
    Test passed

    03/01/2013 4:42:10 PM Unnamed pre-upgrade tests check internal data validity
    Test passed

    03/01/2013 4:42:13 PM Unnamed pre-upgrade tests check event log
    The event log has not been cleared within the last 24 hours.

    03/01/2013 4:42:15 PM Unnamed pre-upgrade tests check event log
    Test passed

    03/01/2013 4:42:15 PM Unnamed pre-upgrade tests pre-upgrade tests complete 0h 0m 11s
    Pre-upgrade tests - all passed

    03/01/2013 4:42:15 PM Unnamed SUPPORT_SERVICES start()
    Thread has been started

    03/01/2013 4:42:15 PM Unnamed array data capture capture storage subsystem data
    Storage Subsystem data collection in progress

    03/01/2013 4:42:30 PM Unnamed SUPPORT_SERVICES ActivateOnlyProgress
    starting activate only

    03/01/2013 4:42:30 PM Unnamed SUPPORT_SERVICES ActivateOnlyProgress
    making facade call to activate firmware

    03/01/2013 4:58:27 PM Unnamed SUPPORT_SERVICES ActivateOnlyProgress
    activation failed due to timeout of controller A
  • SystemAdmin
    SystemAdmin
    4779 Posts

    Re: DS3300 Firmware upgrade failed...

    ‏2013-01-04T00:36:57Z  
    some more info, i have zipped it.
  • SystemAdmin
    SystemAdmin
    4779 Posts

    Re: DS3300 Firmware upgrade failed...

    ‏2013-01-04T01:03:31Z  
    some more info, i have zipped it.
    I have made some progress.

    However, what happened was totally over my head.

    I hope someone can tell me what was going on.

    According to the log above, controller A's activation failed.

    I thought controller B may not have received the activation command.

    Thus I removed controller A and powered on with only B.

    After 15 minutes, I was able to connect to controller B using the SM!!!!!

    I logged in and checked the system, except the complaint about redundancy, everything was ok.

    After that, I connected controller A. Magic happened, it became available after 5 minutes......

    Now it all looked fine....

    What happened....
  • Novikov_Alexander
    Novikov_Alexander
    1404 Posts

    Re: DS3300 Firmware upgrade failed...

    ‏2013-01-04T07:52:54Z  
    I have made some progress.

    However, what happened was totally over my head.

    I hope someone can tell me what was going on.

    According to the log above, controller A's activation failed.

    I thought controller B may not have received the activation command.

    Thus I removed controller A and powered on with only B.

    After 15 minutes, I was able to connect to controller B using the SM!!!!!

    I logged in and checked the system, except the complaint about redundancy, everything was ok.

    After that, I connected controller A. Magic happened, it became available after 5 minutes......

    Now it all looked fine....

    What happened....
    Dear oliver,

    > There were 3 non assigned hard drives in the SAN.

    non-supported configuration used. See info in the DS3300 doc: "Before you power-on the storage subsystem, it must contain at least four drives."

    Regards,
    Alexander Novikov
    Russia, Moscow
  • SystemAdmin
    SystemAdmin
    4779 Posts

    Re: DS3300 Firmware upgrade failed...

    ‏2013-01-07T00:42:04Z  
    Dear oliver,

    > There were 3 non assigned hard drives in the SAN.

    non-supported configuration used. See info in the DS3300 doc: "Before you power-on the storage subsystem, it must contain at least four drives."

    Regards,
    Alexander Novikov
    Russia, Moscow
    Hi Alex,

    Thanks for the reply.

    I think that answered my question.

    The reason I didn't plug in 4 disk was because the raid configure has issue on them.

    Now I have already go things sorted.

    Regards,
    Oliver
  • Novikov_Alexander
    Novikov_Alexander
    1404 Posts

    Re: DS3300 Firmware upgrade failed...

    ‏2013-01-07T19:42:58Z  
    Hi Alex,

    Thanks for the reply.

    I think that answered my question.

    The reason I didn't plug in 4 disk was because the raid configure has issue on them.

    Now I have already go things sorted.

    Regards,
    Oliver
    Hi Oliver,

    you are welcome.

    Regards,
    Alexander Novikov
    Russia, Moscow