Topic
2 replies Latest Post - ‏2013-10-03T10:30:19Z by Wouter Liefting
SystemAdmin
SystemAdmin
6908 Posts
ACCEPTED ANSWER

Pinned topic 2 vios hang when storage DS5020 changes logical drive not on preferred path

‏2012-10-25T21:25:00Z |
I have a problem and I need ideas to resolve. That is the issue:

I have this configurations:

a) 1 power 7 p750 rack server with 1 vios and 1 lpar connected to storage DS5020.
b) 1 power 7 p702 blade server with 1 vios and 1 lpar connected to storage DS5020.
c) 1 power 5 p550 rack server with 1 vios and 1 lpar connected to storage DS5020.

This 3 servers (power7 rack, power 7 blade, power5 rack) are connected to the same storage IBM DS5020.
Reviewing logs at October 12:

The storage DS5020 has this message: Logical drive not on preferred path. 14 logical drives was moved to the second path.

I have errors in 3 VIOS's servers.

In October 12 at 1:57 am, I lost connection to the virtual hdisks and the lpar hangs located in the power 7 p750 rack server.

In October 12 at 2:06 am I lost connection to the virtual hdisks and the lpar hangs located in the power 5 p550 rack server.

In October 12 at 1:59 am I lost connection to the virtual disks and the lpar supports the change in the path located in the power 7 p702 BLADE server. Only this lpar survive and continues working.

The workaround was reboot lpars and VIOS.

Reviewing logs at October 15:

The storage DS5020 has this message: Logical drive not on preferred path. 10 logical drives was moved to the second path.

I have errors in 3 VIOS's servers.

In October 15 at 1:59 am, lpar lost connection to the virtual hdisks and the lpar hangs located in the power 7 p750 rack server.

In October 15 at 2:08 am, lpar lost connection to the virtual hdisks and the lpar hangs located in the power 5 p550 rack server.
The workaround was reboot lpars and VIOS.

Please any ideas?

Attached are the errpt logs.

##############
errpt log in power 7 p750 rack server.

  1. errpt
IDENTIFIER TIMESTAMP T C RESOURCE_NAME DESCRIPTION
DCB47997 1018015812 T H hdisk2 DISK OPERATION ERROR
A6DF45AA 1015043312 I O RMCdaemon The daemon is started.
1BA7DF4E 1015043312 P S SRC SOFTWARE PROGRAM ERROR
CB4A951F 1015043312 I S SRC SOFTWARE PROGRAM ERROR
CB4A951F 1015043312 I S SRC SOFTWARE PROGRAM ERROR
C43F90ED 1015043312 P H hdisk7 SUBSYSTEM COMPONENT FAILURE
2BFA76F6 1015043112 T S SYSPROC SYSTEM SHUTDOWN BY USER
9DBCFDEE 1015043312 T O errdemon ERROR LOGGING TURNED ON
192AC071 1015042812 T O errdemon ERROR LOGGING TURNED OFF
C972F43B 1015042512 T S vhost9 Misbehaved Virtual SCSI Client
0C10BB8C 1015020012 I H hdisk2 ARRAY CONFIGURATION CHANGED
26623394 1015015912 T H fscsi3 COMMUNICATION PROTOCOL ERROR
26623394 1015015912 T H fscsi0 COMMUNICATION PROTOCOL ERROR
D666A8C7 1015015912 T H fcs3 ADAPTER ERROR
D666A8C7 1015015912 T H fcs0 ADAPTER ERROR
A6DF45AA 1012034012 I O RMCdaemon The daemon is started.
1BA7DF4E 1012034012 P S SRC SOFTWARE PROGRAM ERROR
CB4A951F 1012034012 I S SRC SOFTWARE PROGRAM ERROR
CB4A951F 1012034012 I S SRC SOFTWARE PROGRAM ERROR
C43F90ED 1012033912 P H hdisk6 SUBSYSTEM COMPONENT FAILURE
2BFA76F6 1012033812 T S SYSPROC SYSTEM SHUTDOWN BY USER
9DBCFDEE 1012034012 T O errdemon ERROR LOGGING TURNED ON
192AC071 1012033512 T O errdemon ERROR LOGGING TURNED OFF
C972F43B 1012032512 T S vhost14 Misbehaved Virtual SCSI Client
C972F43B 1012032212 T S vhost23 Misbehaved Virtual SCSI Client
C972F43B 1012030512 T S vhost9 Misbehaved Virtual SCSI Client
0C10BB8C 1012015912 I H hdisk12 ARRAY CONFIGURATION CHANGED
DCB47997 1012015812 T H hdisk12 DISK OPERATION ERROR
26623394 1012015812 T H fscsi1 COMMUNICATION PROTOCOL ERROR
26623394 1012015812 T H fscsi2 COMMUNICATION PROTOCOL ERROR
D666A8C7 1012015712 T H fcs1 ADAPTER ERROR
D666A8C7 1012015712 T H fcs2 ADAPTER ERROR
#

##############

##############
errpt log in power 5 p550 rack server.

#
  1. errpt
IDENTIFIER TIMESTAMP T C RESOURCE_NAME DESCRIPTION
A6DF45AA 1015111812 I O RMCdaemon The daemon is started.
1BA7DF4E 1015111712 P S SRC SOFTWARE PROGRAM ERROR
CB4A951F 1015111712 I S SRC SOFTWARE PROGRAM ERROR
CB4A951F 1015111712 I S SRC SOFTWARE PROGRAM ERROR
2BFA76F6 1015111612 T S SYSPROC SYSTEM SHUTDOWN BY USER
9DBCFDEE 1015111712 T O errdemon ERROR LOGGING TURNED ON
192AC071 1015111112 T O errdemon ERROR LOGGING TURNED OFF
C972F43B 1015110912 T S vhost2 Misbehaved Virtual SCSI Client
26623394 1015020812 T H fscsi1 COMMUNICATION PROTOCOL ERROR
D666A8C7 1015020812 T H fcs1 ADAPTER ERROR
0C10BB8C 1012020812 I H hdisk5 ARRAY CONFIGURATION CHANGED
26623394 1012020712 T H fscsi2 COMMUNICATION PROTOCOL ERROR
D666A8C7 1012020612 T H fcs2 ADAPTER ERROR
#

##############

##############
errpt log in power 7 p702 blade server.

  1. errpt
IDENTIFIER TIMESTAMP T C RESOURCE_NAME DESCRIPTION
DCB47997 1018015912 T H hdisk2 DISK OPERATION ERROR
DCB47997 1018015912 T H hdisk3 DISK OPERATION ERROR
4B436A3D 1018015912 T H fscsi1 LINK ERROR
4B436A3D 1018015912 T H fscsi1 LINK ERROR
DC73C03A 1015020112 T S fscsi0 SOFTWARE PROGRAM ERROR
0C10BB8C 1012035812 I H hdisk6 ARRAY CONFIGURATION CHANGED
0C10BB8C 1012035812 I H hdisk6 ARRAY CONFIGURATION CHANGED
0C10BB8C 1012035512 I H hdisk6 ARRAY CONFIGURATION CHANGED
0C10BB8C 1012035512 I H hdisk6 ARRAY CONFIGURATION CHANGED
0C10BB8C 1012035212 I H hdisk6 ARRAY CONFIGURATION CHANGED
0C10BB8C 1012035212 I H hdisk6 ARRAY CONFIGURATION CHANGED
0C10BB8C 1012034912 I H hdisk6 ARRAY CONFIGURATION CHANGED
0C10BB8C 1012034912 I H hdisk6 ARRAY CONFIGURATION CHANGED
0C10BB8C 1012034612 I H hdisk6 ARRAY CONFIGURATION CHANGED
0C10BB8C 1012034612 I H hdisk6 ARRAY CONFIGURATION CHANGED
0C10BB8C 1012034312 I H hdisk6 ARRAY CONFIGURATION CHANGED
0C10BB8C 1012034312 I H hdisk6 ARRAY CONFIGURATION CHANGED
0C10BB8C 1012034012 I H hdisk6 ARRAY CONFIGURATION CHANGED
0C10BB8C 1012034012 I H hdisk6 ARRAY CONFIGURATION CHANGED
0C10BB8C 1012033712 I H hdisk6 ARRAY CONFIGURATION CHANGED
0C10BB8C 1012033712 I H hdisk6 ARRAY CONFIGURATION CHANGED
0C10BB8C 1012033412 I H hdisk6 ARRAY CONFIGURATION CHANGED
0C10BB8C 1012033412 I H hdisk6 ARRAY CONFIGURATION CHANGED
0C10BB8C 1012033312 I H hdisk5 ARRAY CONFIGURATION CHANGED
0C10BB8C 1012033312 I H hdisk5 ARRAY CONFIGURATION CHANGED
0C10BB8C 1012033112 I H hdisk4 ARRAY CONFIGURATION CHANGED
0C10BB8C 1012033112 I H hdisk4 ARRAY CONFIGURATION CHANGED
0C10BB8C 1012033112 I H hdisk3 ARRAY CONFIGURATION CHANGED
0C10BB8C 1012032512 I H hdisk2 ARRAY CONFIGURATION CHANGED
0C10BB8C 1012032412 I H hdisk1 ARRAY CONFIGURATION CHANGED
DCB47997 1012020012 T H hdisk2 DISK OPERATION ERROR
DCB47997 1012020012 T H hdisk2 DISK OPERATION ERROR
0C10BB8C 1012020012 I H hdisk2 ARRAY CONFIGURATION CHANGED
0C10BB8C 1012020012 I H hdisk3 ARRAY CONFIGURATION CHANGED
0C10BB8C 1012020012 I H hdisk4 ARRAY CONFIGURATION CHANGED
0C10BB8C 1012020012 I H hdisk5 ARRAY CONFIGURATION CHANGED
0C10BB8C 1012020012 I H hdisk1 ARRAY CONFIGURATION CHANGED
0C10BB8C 1012020012 I H hdisk2 ARRAY CONFIGURATION CHANGED
DC73C03A 1012015912 T S fscsi1 SOFTWARE PROGRAM ERROR
4B436A3D 1012015912 T H fscsi1 LINK ERROR
4B436A3D 1012015912 T H fscsi1 LINK ERROR
4B436A3D 1012015912 T H fscsi1 LINK ERROR

##############
  • DGHM_Johannes_Loebel
    1 Post
    ACCEPTED ANSWER

    Re: 2 vios hang when storage DS5020 changes logical drive not on preferred path

    ‏2013-09-27T15:39:38Z  in response to SystemAdmin

    I know this is a old topic but i have the same problem.

    Do you receive some help or hint about this?

     

     

  • Wouter Liefting
    Wouter Liefting
    54 Posts
    ACCEPTED ANSWER

    Re: 2 vios hang when storage DS5020 changes logical drive not on preferred path

    ‏2013-10-03T10:30:19Z  in response to SystemAdmin

    Have you checked that you have applied the SCSI "no reserve" policy consistently throughout?

    On AIX: chdev -l hdiskn -a reserve_policy=no_reserve

    On VIOS: chdev -dev hdiskn -attr reserve_policy=no_reserve

    If the disks are active you might need to do a reboot to activate this feature. You can't change the reservation policy after a disk has become active, since the reservation is done when the disk becomes active.

    Another thing to check is that your SAN zoning is implemented properly throughout the SAN.