Topic
  • 5 replies
  • Latest Post - ‏2012-03-01T12:01:49Z by Crazyman
Crazyman
Crazyman
3 Posts

Pinned topic hdiskX in VIOC and hdiskpowerX VIOS mapping relationship with MPIO

‏2012-02-25T08:50:16Z |
Dear,
the config of my environment is:
dual VIOS
vscsi for VIOC
EMC DMX storage and powerpath software for VIOS
We have 2 storages, STG1 and STG2.

LUN1 is from STG1, and LUN2 is from STG2.
both LUN1 and LUN2 assigned to VIOS1 and VIOS2, in the VIOS side, the two LUNs recognized as hdiskpower0 and hdiskpower3.
both LUNs mapped to VIOC ( vhost0 ), thus in VIOC, the LUNs recognized as hdisk0 and hdisk1.
My question is:
How to determine the backing device of hdisk0: is it LUN1 or LUN2, i.e., hdiskpower0 or hdiskpower3?

Note:
1. the PVID method can't solve, since PVID may change in VIOC when perform chdev -l hdisk0 pv=clear/yes.
2. VIOC LUN id also doesn't work. We have experienced:
in VIOC, hdisk0 LUN id is like: L8100000000000000
in VIOS1, the LUN id of backing device (LUN1) is like: 8100000000000000
BUT in VIOS2, the LUN id of backing device (also LUN1 from the same storage) is like: 8200000000000000
on the VIOC side:
vscsi0 is mapped to VIOS1 and vscsi1 is mapped to VIOS2.
Now let's discuss vscsi0 only. -vpl vscsi*
vscsi1 U9117.MMB.06XXXXX-V3-C3-T1 Virtual SCSI Client Adapter
root@testlpar/tmp#lscfg
Hardware Location Code......U9117.MMB.06XXXXX-V3-C3-T1
vscsi0 U9117.MMB.06XXXXX-V3-C2-T1 Virtual SCSI Client Adapter
Hardware Location Code......U9117.MMB.06XXXXX-V3-C2-T1

PLATFORM SPECIFIC
Name: v-scsi
Node: v-scsi@30000002
Device Type: vscsi
Physical Location: U9117.MMB.06XXXXX-V3-C2-T1
Name: v-scsi
Node: v-scsi@30000003
Device Type: vscsi
Physical Location: U9117.MMB.06XXXXX-V3-C3-T1

vscsi0 can see both hdisk0 and hdisk1# lspath
Enabled hdisk0 vscsi0
Enabled hdisk1 vscsi0
Enabled hdisk0 vscsi1
Enabled hdisk1 vscsi1# lsdev -Cc disk|grep SCSI
hdisk0 Available Virtual SCSI Disk Drive
hdisk1 Available Virtual SCSI Disk Drive
root@testlpar/tmp
root@testlpar/tmp

root@testlpar/#lscfg -vpl hdisk0
hdisk0 U9117.MMB.06XXXXX-V3-C2-T1-L8100000000000000 Virtual SCSI Disk Drive
PLATFORM SPECIFIC
Name: disk
Node: disk
Device Type: block
root@testlpar/#lscfg -vpl hdisk1
hdisk1 U9117.MMB.06XXXXX-V3-C2-T1-L8200000000000000 Virtual SCSI Disk Drive
PLATFORM SPECIFIC
Name: disk
Node: disk
Device Type: block

On the VIOS side:
both LUNs mapped to vhost0

<padmin@XXXXX_VIOS1:/home/padmin>$ lsmap -vadapter vhost0
SVSA Physloc Client Partition ID

--------------------------------------------
vhost0 U9117.MMB.06XXXXX-V1-C103 0x00000003
VTD rg0_testlpar
Status Available
LUN 0x8100000000000000
Backing device hdiskpower0
Physloc U78C0.001.DBJM630-P2-C1-T1-L3
Mirrored false
VTD rg1_testlpar
Status Available
LUN 0x8200000000000000
Backing device hdiskpower3
Physloc U78C0.001.DBJM630-P2-C1-T1-L7
Mirrored false
Updated on 2012-03-01T12:01:49Z at 2012-03-01T12:01:49Z by Crazyman
  • jvk
    jvk
    10 Posts

    Re: hdiskX in VIOC and hdiskpowerX VIOS mapping relationship with MPIO

    ‏2012-02-26T15:37:15Z  
    You can look the "unique_id" of your hdisk/hdiskpower parameter on VIOC/VIOS and compare them. This works for me but I think this is not the only way..
    The "unique_id" of hdiskpowerN on VIOS is part of the "unique_id" of hdiskM on VIOC.

    Example: find a corresponding hdiskpowerN on VIOS for the hdisk0 disk on VIOC:

    VIOC:

    odmget CuAt | grep -p hdisk0 | grep -p unique_id

    CuAt:
    name = "hdisk0"
    attribute = "unique_id"
    value = "321D1D0690211109SYMMETRIX03EMCfcp05VDASD03AIXvscsi"
    ______________****************************

    Take "1D0690211109SYMMETRIX03EMCfcp" from above and go to VIOS:

    VIOS:

    odmget CuAt | grep -p 1D0690211109SYMMETRIX03EMCfcp | grep -p unique_id

    CuAt:
    name = "hdiskpower0"
    attribute = "unique_id"
    value = "1D0690211109SYMMETRIX03EMCfcp"

    So hdisk0 on VIOC maps to hdiskpower0 on VIOS.
    In dual VIOS, do the same on second VIOS. hdiskpower number on each VIOS is not necessary the same (means hdiskpower0 on VIOS 1 in not necessary the same LUN as hdiskpower0 on VIOS 2).
  • krichard
    krichard
    2 Posts

    Re: hdiskX in VIOC and hdiskpowerX VIOS mapping relationship with MPIO

    ‏2012-02-27T14:30:36Z  
    If you want to determine the backing device relationship you can run the following command from the VIOS. You will need to have the EMC ODM definitions instaled or you can get the INQ utility from the ftp ftp://ftp.emc.com/pub/elab/inquiry/v7.3.2.0/
    /usr/lpp/EMC/Symmetrix/bin/inq.aix64_51 -emcvdasd
  • Crazyman
    Crazyman
    3 Posts

    Re: hdiskX in VIOC and hdiskpowerX VIOS mapping relationship with MPIO

    ‏2012-03-01T06:31:13Z  
    • jvk
    • ‏2012-02-26T15:37:15Z
    You can look the "unique_id" of your hdisk/hdiskpower parameter on VIOC/VIOS and compare them. This works for me but I think this is not the only way..
    The "unique_id" of hdiskpowerN on VIOS is part of the "unique_id" of hdiskM on VIOC.

    Example: find a corresponding hdiskpowerN on VIOS for the hdisk0 disk on VIOC:

    VIOC:

    odmget CuAt | grep -p hdisk0 | grep -p unique_id

    CuAt:
    name = "hdisk0"
    attribute = "unique_id"
    value = "321D1D0690211109SYMMETRIX03EMCfcp05VDASD03AIXvscsi"
    ______________****************************

    Take "1D0690211109SYMMETRIX03EMCfcp" from above and go to VIOS:

    VIOS:

    odmget CuAt | grep -p 1D0690211109SYMMETRIX03EMCfcp | grep -p unique_id

    CuAt:
    name = "hdiskpower0"
    attribute = "unique_id"
    value = "1D0690211109SYMMETRIX03EMCfcp"

    So hdisk0 on VIOC maps to hdiskpower0 on VIOS.
    In dual VIOS, do the same on second VIOS. hdiskpower number on each VIOS is not necessary the same (means hdiskpower0 on VIOS 1 in not necessary the same LUN as hdiskpower0 on VIOS 2).
    Thanks a lot!

    still two questions:
    1.
    in VIOC using ibm native MPIO, what does the driver use to determine multi-path and for disk identification? that's, how can it determine the two paths are in fact one disk?

    2.
    unique_id can read from the ODM of VIOC or VIOS, what is unique_id of a disk related to? the LUN serial of time stamp? in what conditions will it change? at what time is it generated?
    as we know, pvid is generated with combination of time stamp, thus when used "chdev -l hdisk0 -a pv=clear / yes ", the PVID of hdisk0 changes.
  • jvk
    jvk
    10 Posts

    Re: hdiskX in VIOC and hdiskpowerX VIOS mapping relationship with MPIO

    ‏2012-03-01T10:24:32Z  
    • Crazyman
    • ‏2012-03-01T06:31:13Z
    Thanks a lot!

    still two questions:
    1.
    in VIOC using ibm native MPIO, what does the driver use to determine multi-path and for disk identification? that's, how can it determine the two paths are in fact one disk?

    2.
    unique_id can read from the ODM of VIOC or VIOS, what is unique_id of a disk related to? the LUN serial of time stamp? in what conditions will it change? at what time is it generated?
    as we know, pvid is generated with combination of time stamp, thus when used "chdev -l hdisk0 -a pv=clear / yes ", the PVID of hdisk0 changes.
    Multipathing info for mpio disks is stored in CuPath & CuPathAt ODM object classes. Run the 'lspath -F "name parent path_id status connection" -l hdiskN' command and see multipahting config for your hdiksN. Or in short, the lspath only.

    The unique_id is composed at config time and as you can see, it contains the LUN SN, product and vendor id, the udid type and numbers which tell us the length of mentioned fileds. The unique_id can change after reconfiguration when the LUN SN changes.
  • Crazyman
    Crazyman
    3 Posts

    Re: hdiskX in VIOC and hdiskpowerX VIOS mapping relationship with MPIO

    ‏2012-03-01T12:01:49Z  
    • jvk
    • ‏2012-03-01T10:24:32Z
    Multipathing info for mpio disks is stored in CuPath & CuPathAt ODM object classes. Run the 'lspath -F "name parent path_id status connection" -l hdiskN' command and see multipahting config for your hdiksN. Or in short, the lspath only.

    The unique_id is composed at config time and as you can see, it contains the LUN SN, product and vendor id, the udid type and numbers which tell us the length of mentioned fileds. The unique_id can change after reconfiguration when the LUN SN changes.
    except the unique_id method, i can also follow this process to determine the backing device of vscsi:

    oprist/#lspath -F "name parent path_id status connection" -l hdisk1
    hdisk1 vscsi0 0 Enabled 820000000000
    hdisk1 vscsi1 1 Enabled 830000000000

    oprist/#lscfg -vpl vscsi0
    vscsi0 U8233.E8B.06E7CBP-V24-C2-T1 Virtual SCSI Client Adapter

    Hardware Location Code......U8233.E8B.06E7CBP-V24-C2-T1

    in HMC, i can know vscsi0 is mapped to vhost21 in VIOS1

    in VIOS1:

    <E7CBP_VIOS1:/home/padmin>$ lsmap -vadapter vhost21
    SVSA Physloc Client Partition ID

    --------------------------------------------
    vhost21 U8233.E8B.06E7CBP-V1-C124 0x00000018

    VTD r0_oprist
    Status Available
    LUN 0x8100000000000000
    Backing device hdiskpower19
    Physloc U5802.001.9K8H537-P1-C7-T1-L20
    Mirrored false

    VTD rgtest0
    Status Available
    LUN 0x8200000000000000
    Backing device hdiskpower36
    Physloc U5802.001.9K8H537-P1-C7-T1-L76
    Mirrored false

    VTD rgtest1
    Status Available
    LUN 0x8300000000000000
    Backing device hdiskpower37
    Physloc U5802.001.9K8H537-P1-C7-T1-L77
    Mirrored false

    through
    "hdisk1 vscsi0 0 Enabled 820000000000" in the lspath output, i can determine the backing device is hdiskpower36.

    Does this process make sense?
    By the way, there are still a few questions:

    1. How to read PVID directly from disk? when using "lquerypv -H /dev/hdisk0", i get a problem:
    in normal environment, this command runs well. but after i "chdev -l hdiskpowerX -a pv=clear/yes" in VIOS, the PVID in the backing disk changes actually. then, i run "lquerypv -H /dev/hdisk0" in VIOC again, only get nothing output. in this condition, how to get the PVID in the VIOC side. Note, i can't delete the hdisk in VIOC.

    2. i still don't know how the "unique_id" of a disk generate. in my experiment, i assigned the same LUN to other system, and get the same unique_id. It seems nothing to do with timestamp. It doesn't contain the LUN SN. In what condition will it change?
    Would you please explain more in detail about unique_id?

    Thanks so much!

    <E7CBP_VIOS1:/home/padmin>$ powermt display dev=hdiskpower36
    Pseudo name=hdiskpower36
    CLARiiON ID=CKM00084900113 SG_zzq_e7cbp_vios1_vios2
    Logical device ID=600601608D502200AC25AB036663E111 LUN 1305
    state=alive; policy=CLAROpt; priority=0; queued-IOs=0
    Owner: default=SP A, current=SP A Array failover mode: 3
    ==============================================================================

    Host
    - Stor - -- I/O Path - -- Stats ---
    1. HW Path I/O Paths Interf. Mode State Q-IOs Errors
    ==============================================================================
    0 fscsi0 hdisk146 SP A0 active alive 0 0
    0 fscsi0 hdisk148 SP B0 active alive 0 0
    1 fscsi2 hdisk150 SP A0 active alive 0 0
    1 fscsi2 hdisk152 SP B0 active alive 0 0

    <E7CBP_VIOS1:/home/padmin>$ odmget -q name=hdiskpower36 CuAt|grep -p unique_id
    CuAt:
    name = "hdiskpower36"
    attribute = "unique_id"
    value = "35213600601608D502200AC25AB036663E11106RAID 503DGCfcp"
    type = "R"
    generic = ""
    rep = "nl"
    nls_index = 124

    oprist/#odmget -q name=hdisk1 CuAt |grep -p unique_id
    CuAt:
    name = "hdisk1"
    attribute = "unique_id"
    value = "4A3535213600601608D502200AC25AB036663E11106RAID 503DGCfcp05VDASD03AIXvscsi"
    type = "R"
    generic = ""
    rep = "n"
    nls_index = 0