• 3 replies
  • Latest Post - ‏2014-06-13T07:13:30Z by HajoEhlers
5 Posts

Pinned topic best practice for adding/removing NSD servers?

‏2014-05-30T21:02:04Z |

Very similar to the thread on "change order of round-robin nsd striped writes", I'm looking for recommended practices for adding & removing NSD servers. Specifically, I've got a GPFS cluster under Linux with 3 NSD servers. I intend to add 3~4 more servers, then eventually retire the initial 3 machines.

The NSD servers are attached to a fibre SAN, and have ethernet connections to 50+ GPFS clients which are not SAN-attached. A few machines in our environment use NFS to access data, exported via cNFS from the GPFS servers.

My concern is that the disk stanzas used by mmcrnsd, mmadddisk, etc. have an explicit server list.

Are there recommended practices for adding GPFS servers to an existing environment, including updating the server list, and recommendations for removing the older GPFS servers, all while causing the least disruption to clients of the filesystem? I would strongly prefer not to bring down the filesystems if possible.

  • fleers
    26 Posts

    Re: best practice for adding/removing NSD servers?


    Once you get your new NSD servers attached to the FC storage and added to the cluster, use the mmchnsd command to change the serverlist for the NSDs in question(after editing the stanza file as needed).  You can do this online.  You may want to invoke mmchnsd twice - once when you add the new NSD servers, then after you are comfortable with the new NSD servers performing as expected, to delete the retiring servers from the serverlist.

    If you are removing configuration servers from the cluster see mmchcluster.  Make sure you maintain quorum when removing old NSD servers (mmdelnode).

    Updated on 2014-06-02T17:28:17Z at 2014-06-02T17:28:17Z by fleers
  • esj
    104 Posts

    Re: best practice for adding/removing NSD servers?


    The following is already mentioned in the man page but maybe its worth repeating one more time

    You can change the NSD servers only with the mmchnsd command.  Note that you cannot add or delete individual NSD server nodes.  You need to specify the entire list even if you are changing only a single name.

    Note also that the file system to which the NSD belongs must be unmounted.  GPFS cannot currently handle dynamic changes to the NSD server list.  In that sense, the command is disruptive and requires some file system down time.

  • HajoEhlers
    254 Posts

    Re: best practice for adding/removing NSD servers?


    In case you have enough storage.....

    • Create new NSD server
    • Create new LUNs
    • Assign the new luns to the new NSD server - The new NSD server DO NOT serve the old luns
    • Assigne the new luns to your gpfs - Now the old luns are serverd by the old nsd server and the new luns by the new nsd server
    • Migrate from the old nsd server the data to the new nsd server
    • As soon as all old luns( nsd disks )  on the old nsd server are you empty - remove the empty nsd disk  and then remove the old nsd server.