Topic
  • 4 replies
  • Latest Post - ‏2009-03-23T14:31:56Z by Theeraph
Theeraph
Theeraph
110 Posts

Pinned topic GPFS 3.2.1.10-How to change vgname in lspv output to the correct NSD name?

‏2009-03-19T15:43:25Z |
Hi,

I have configured a 2-node GPFS 3.2.1.10 cluster. Then the customer wants to reinstall 1 node (node 1), so I mmdelnode it then reinstall AIX on it and install GPFS, then mmaddnode.

Now everything seems to be normal except one thing 'lspv does not show the correct NSD name': (I noticed that the vgname is gpfs since I have completed AIX re-installation, I expected it to change after mmaddnode, but it didn't.)

ICSPRC-DB-T1:root:/> mmlscluster

GPFS cluster information
========================
GPFS cluster name: ICS_IWT_cluster.tgpfs1
GPFS cluster id: 13882420095409259527
GPFS UID domain: ICS_IWT_domain
Remote shell command: /usr/bin/ssh
Remote file copy command: /usr/bin/scp

GPFS cluster configuration servers:

Primary server: ICSPRC-GPFS-T2
Secondary server: ICSPRC-GPFS-T1

Node Daemon node name IP address Admin node name
Designation


2 ICSPRC-GPFS-T2 192.168.67.135 ICSPRC-GPFS-T2
quorum-manager
3 ICSPRC-GPFS-T1 192.168.67.134 ICSPRC-GPFS-T1
quorum-manager
.
ICSPRC-DB-T1:root:/> mmgetstate -La

Node number Node name Quorum Nodes up Total nodes GPFS state Remarks


----
2 ICSPRC-GPFS-T2 1* 2 2 active quorum
node
3 ICSPRC-GPFS-T1 1* 2 2 active quorum
node
.
ICSPRC-DB-T1:root:/> mmlsconfig
Configuration data for cluster ICS_IWT_cluster.tgpfs1:

clusterName ICS_IWT_cluster.tgpfs1
clusterId 13882420095409259527
clusterType lc
autoload yes
uidDomain ICS_IWT_domain
minReleaseLevel 3.2.1.5
dmapiFileHandleSize 32
pagepool 100M
maxMBpS 512
prefetchThreads 100
worker1Threads 449
tiebreakerDisks ics01disk;ics02disk;ics03disk

File systems in cluster ICS_IWT_cluster.tgpfs1:

/dev/gpfs
.
ICSPRC-DB-T1:root:/> mmlsnsd

File system Disk name NSD servers

gpfs ics01disk (directly attached)
gpfs ics02disk (directly attached)
gpfs ics03disk (directly attached)
.
ICSPRC-DB-T1:root:/> mmdf gpfs
disk disk size failure holds holds free KB
free KB
name in KB group metadata data in full blocks
in fragments

-------------
--------
--------------------

Disks in storage pool: system (Maximum disk size allowed is 101 GB)
ics01disk 5242880 -1 yes yes 4564992 ( 87%)
4288 ( 0%)
ics02disk 5242880 -1 yes yes 4564480 ( 87%)
1744 ( 0%)
ics03disk 5242880 -1 yes yes 4565504 ( 87%)
2736 ( 0%)


-----

(pool total) 15728640 13694976 ( 87%)
8768 ( 0%)

============= ==================== =====
==============
(total) 15728640 13694976 ( 87%)
8768 ( 0%)

Inode Information

Number of used inodes: 4061
Number of free inodes: 29731
Number of allocated inodes: 33792
Maximum number of inodes: 33792
.
ICSPRC-DB-T1:root:/> lspv
hdisk0 00cec7138ab22dbb rootvg active
hdisk1 00cec713889cfc9b rootvg active
hdisk2 none None
hdisk3 none None
hdisk4 none None
hdisk5 none None
hdisk6 none None
hdisk7 00cec743ac166c37 gpfs
hdisk8 00cec743ac166d01 gpfs
hdisk9 00cec743ac166da9 gpfs
.
(lspv on the other node shows the correct NSD names.)
.
1. How can I change the name 'gpfs' in lspv output to the correct name?
.
2. Is it possible for me to change the newly added node node number from 3 to 1? (I was surprised that GPFS does not reuse node number 1...)?
.
Thank you very much,
Theeraphong
Updated on 2009-03-23T14:31:56Z at 2009-03-23T14:31:56Z by Theeraph
  • esj
    esj
    104 Posts

    Re: GPFS 3.2.1.10-How to change vgname in lspv output to the correct NSD name?

    ‏2009-03-21T17:53:39Z  
    To force the lspv information to be refreshed, just remove file
    /var/mmfs/gen/nsdpvol and then run mmlspv. You should now see
    the the correct NSD names. If mmlspv shows the correct stuff,
    but you still get the wrong answer from lspv, you need to talk
    to the AIX guys.

    To change the node number, provided you are running GPFS 3.2, run
    mmshutdown -N all
    mmchnode --node-number=NN -N nodeName
    Note that this command requires GPFS to be stopped on all nodes.
    You can also issue "mmchnode --node-number=DEFAULT -N all" and
    GPFS will renumber all nodes from 1 to N.

    The two main reasons for not reusing the node numbers are that
    this is essentially a "cosmetic" attribute; the important thing
    is that the numbers be unique within a cluster. And, a lot more
    important, reusing node numbers would prevent you from being
    able to add or delete nodes without first shutting down GPFS
    on all nodes in the cluster.

    Hope this helps,
    Eugene
  • Theeraph
    Theeraph
    110 Posts

    Re: GPFS 3.2.1.10-How to change vgname in lspv output to the correct NSD name?

    ‏2009-03-22T02:03:28Z  
    • esj
    • ‏2009-03-21T17:53:39Z
    To force the lspv information to be refreshed, just remove file
    /var/mmfs/gen/nsdpvol and then run mmlspv. You should now see
    the the correct NSD names. If mmlspv shows the correct stuff,
    but you still get the wrong answer from lspv, you need to talk
    to the AIX guys.

    To change the node number, provided you are running GPFS 3.2, run
    mmshutdown -N all
    mmchnode --node-number=NN -N nodeName
    Note that this command requires GPFS to be stopped on all nodes.
    You can also issue "mmchnode --node-number=DEFAULT -N all" and
    GPFS will renumber all nodes from 1 to N.

    The two main reasons for not reusing the node numbers are that
    this is essentially a "cosmetic" attribute; the important thing
    is that the numbers be unique within a cluster. And, a lot more
    important, reusing node numbers would prevent you from being
    able to add or delete nodes without first shutting down GPFS
    on all nodes in the cluster.

    Hope this helps,
    Eugene
    Hi,

    ! To force the lspv information to be refreshed, just remove file /var/mmfs/gen/nsdpvol and then run mmlspv. You should now see the the correct NSD names.

    So it looks like nsdpvol is keeping the old NSD name, and lspv uses this name.

    Once nsdpvol file is removed, mmlspv will scan the disk (and thus get the correct name) and create a new nsdpvol file with the new name.

    lspv then should show the correct name.

    if mmlspv shows the correct name (sorry, I did not try mmlspv on any node, so I do not know it), but lspv is still wrong, then something is not right with lspv...

    Do I understand correctly?

    Thank you very much,
    Theeraphong
  • esj
    esj
    104 Posts

    Re: GPFS 3.2.1.10-How to change vgname in lspv output to the correct NSD name?

    ‏2009-03-22T05:36:23Z  
    • Theeraph
    • ‏2009-03-22T02:03:28Z
    Hi,

    ! To force the lspv information to be refreshed, just remove file /var/mmfs/gen/nsdpvol and then run mmlspv. You should now see the the correct NSD names.

    So it looks like nsdpvol is keeping the old NSD name, and lspv uses this name.

    Once nsdpvol file is removed, mmlspv will scan the disk (and thus get the correct name) and create a new nsdpvol file with the new name.

    lspv then should show the correct name.

    if mmlspv shows the correct name (sorry, I did not try mmlspv on any node, so I do not know it), but lspv is still wrong, then something is not right with lspv...

    Do I understand correctly?

    Thank you very much,
    Theeraphong
    mlsmspv is a small script that is used by the AIX lspv command to find out which disks have been formatted as GPFS NSDs. In turn, mmlspv retrieves this information from /var/mmfs/gen/nsdpvol. This file is essentially a cache of all locally accessible disks that have been formatted as NSDs. If the mmlspv script cannot find the file, it will re-scan the local disks and recreate it.

    Bottom line, it is safe to remove /var/mmfs/gen/nsdpvol and then run lspv.
  • Theeraph
    Theeraph
    110 Posts

    Re: GPFS 3.2.1.10-How to change vgname in lspv output to the correct NSD name?

    ‏2009-03-23T14:31:56Z  
    • esj
    • ‏2009-03-22T05:36:23Z
    mlsmspv is a small script that is used by the AIX lspv command to find out which disks have been formatted as GPFS NSDs. In turn, mmlspv retrieves this information from /var/mmfs/gen/nsdpvol. This file is essentially a cache of all locally accessible disks that have been formatted as NSDs. If the mmlspv script cannot find the file, it will re-scan the local disks and recreate it.

    Bottom line, it is safe to remove /var/mmfs/gen/nsdpvol and then run lspv.
    Eugene,

    Thank you very much for your advices,
    Theeraphong