mmaddnode command

Adds nodes to a GPFS cluster.

Synopsis

mmaddnode -N {NodeDesc[,NodeDesc...] | NodeFile} [--accept] [--on-error-continue]

Availability

Available on all IBM Storage Scale editions.

Description

Use the mmaddnode command to add nodes to an existing IBM Storage Scale cluster. On each new node, a mount point directory and character mode device is created for each GPFS file system.

Follow these rules when adding nodes to an IBM Storage Scale cluster:

You may issue the command only from a node that already belongs to the IBM Storage Scale cluster.
IBM Storage Scale must be installed on the nodes before you run the mmaddnode command to add those nodes to the cluster.
While a node may mount file systems from multiple clusters, the node itself may only be added to a single cluster using the mmcrcluster or mmaddnode command.
The nodes must be available for the command to be successful. If any of the nodes listed are not available when the command is issued, a message listing those nodes is displayed. You must correct the problem on each node and reissue the command to add those nodes.
After the nodes are added to the cluster, use the mmchlicense command to designate appropriate IBM Storage Scale licenses to the new nodes.

Note: The mmaddnode command is not supported after an IBM Storage Scale native REST API cluster is created or an existing cluster is migrated to IBM Storage Scale native REST API. For more information, see Limitations of IBM Storage Scale native REST API.

Parameters

-N NodeDesc[,NodeDesc...] | NodeFile

Specifies node descriptors, which provide information about nodes to be added to the cluster.

NodeFile

Specifies a file containing a list of node descriptors, one per line, to be added to the cluster.

NodeDesc[,NodeDesc...]

Specifies the list of nodes and node designations to be added to the IBM Storage Scale cluster. Node descriptors are defined as:

NodeName:NodeDesignations:AdminNodeName:LicenseType:NodeComment

where:

NodeName

Specifies the host name or IP address of the node for GPFS daemon-to-daemon communication.

The host name or IP address must refer to the communication adapter over which the GPFS daemons communicate. Aliased interfaces are not allowed. Use the original address or a name that is resolved by the host command to that original address. You can specify a node using any of these forms:

Short host name (for example, h135n01)
Long, fully qualified, host name (for example, h135n01.ibm.com)
IP address (for example, 7.111.12.102). IPv6 addresses must be enclosed in brackets (for example, [2001:192::192:168:115:124]).

Regardless of which form you use, GPFS will resolve the input to a host name and an IP address and will store these in its configuration files. It is expected that those values will not change while the node belongs to the cluster.

NodeDesignations

An optional, "-" separated list of node roles:

manager | client – Indicates whether a node is part of the node pool from which file system managers and token managers can be selected. The default is client.
quorum | nonquorum – Indicates whether a node is counted as a quorum node. The default is nonquorum.
Note: If you are designating a new node as a quorum node, and adminMode central is in effect for the cluster, GPFS must be down on all nodes in the cluster. Alternatively, you may choose to add the new nodes as nonquorum and once GPFS has been successfully started on the new nodes, you can change their designation to quorum using the mmchnode command.
perfmon - Designates a node as a performance monitoring (perfmon) node. The perfmon designation is ignored if the node is not a Linux® node or if the performance monitoring configuration has not been setup yet. For more details about how to generate the configuration of the performance monitoring tool, see mmperfmon command.

AdminNodeName

Specifies an optional field that consists of a node interface name to be used by the administration commands to communicate between nodes. If AdminNodeName is not specified, the NodeName value is used.

Note: AdminNodeName must be a resolvable network host name. For more information, see GPFS node adapter interface names.

LicenseType

Assigns a license of the specified type to the node. Valid values are server, client, and fpo. For information about these license types, see mmchlicense command.

The full text of the Licensing Agreement is provided with the installation media and can be found at the IBM® Software license agreements website (www.ibm.com/software/sla/sladb.nsf).

NodeComment

Specifies an optional field that provides additional information on the node. The comment field might also be included in the NodeFile file as a parameter.

Note:

Comments can only contain the following characters:
0-9 A-Z a-z (space) # - . @ _
While running a command with the -N option or using a NodeFile attribute to provide multiple node descriptors, all comments that include one or more spaces must be enclosed within single quotation marks (') or double quotation marks ("). You must always use quotation marks with comments to avoid possible parsing errors.
Comments must not begin with a -.
Comments cannot be longer than 32 characters.
Comments that include spaces at the beginning or end will have the spaces trimmed.

The comment field adds new data that is stored in the cluster configuration file and is available only to nodes running on IBM Storage Scale 5.1.4 or later for reading and configuring.

--accept

Specifies that you accept the terms of the applicable license agreement. If one or more node descriptors includes a LicenseType term, this parameter causes the command to skip the prompt for you to accept a license.

--on-error-continue

Enables the command to trap invalid nodes, remove those nodes from the node list, and continue to process the command. Without selecting this option, the command may exit for any reason when a node is found to be invalid.

Note: Not all error conditions relating to a node or the program function can allow the command to continue.

When this option is used, the add node process runs in two stages and the progress of each stage is displayed on the console as it runs. The --on-error-continue option provides the following functions:

Some errors are trapped and the nodes that generated the errors are removed from the list of valid nodes to process. This allows the program to continue as long as there is at least one valid node remaining in the add node process.
Presents the console output in more of a report format.
Creates two stages in the add node process. The first stage is where some node errors can be trapped and thus the node gets excluded from the process. The second stage is more critical and will error out on any significant errors.
In stage one, if nodes failed to get added to the cluster, two files are created to list the valid nodes and invalid nodes separately. The user gets informed of these files. This allows the user to rerun with just the valid node list.

Refer to Example 5 for the example of the add node process with the --on-error-continue option enabled.

You must provide a NodeDesc for each node to be added to the IBM Storage Scale cluster.

Exit status

0: Successful completion.
nonzero: A failure has occurred.

Security

You must have root authority to run the mmaddnode command.

The node on which the command is issued must be able to execute remote shell commands on any other node in the cluster without the use of a password and without producing any extraneous messages. For more information, see Requirements for administering a GPFS file system.

Examples

To add nodes k164n06 and k164n07 as quorum nodes, designating k164n06 to be available as a manager node, issue the following command:

# mmaddnode -N k164n06:quorum-manager,k164n07:quorum

To confirm the addition, issue the following command:

# mmlscluster

A sample output is as follows:

GPFS cluster information
========================
  GPFS cluster name:       cluster1.kgn.ibm.com
  GPFS cluster id:         680681562214606028
  GPFS UID domain:         cluster1.kgn.ibm.com
  Remote shell command:      /usr/bin/ssh
  Remote file copy command:  /usr/bin/scp
  Repository type:           server-based

GPFS cluster configuration servers:
-----------------------------------
  Primary server:    k164n07.kgn.ibm.com
  Secondary server:  k164n04.kgn.ibm.com

Node Daemon node name      IP address    Admin node name  Designation
---------------------------------------------------------------------
 1   k164n04.kgn.ibm.com   198.117.68.68  k164n04.kgn.ibm.com  quorum
 2   k164n07.kgn.ibm.com   198.117.68.71  k164n07.kgn.ibm.com  quorum
 3   k164n06.kgn.ibm.com   198.117.68.70  k164n06.kgn.ibm.com  quorum-manager

In the following example the mmaddnode command adds a node without specifying a license:

(11:37:06) c34f2n03:~ # mmaddnode -N c6f2bc4n8:quorum
Tue Mar 12 11:37:13 EDT 2019: mmaddnode: Processing node c6f2bc4n8.gpfs.net
mmaddnode: Command successfully completed
mmaddnode: Warning: Not all nodes have proper GPFS license designations.
    Use the mmchlicense command to designate licenses as needed.
mmaddnode: Propagating the cluster configuration data to all
  affected nodes.  This is an asynchronous process.

The command displays a warning message that some nodes do not have licenses.

In the following example the mmaddnode command specifies a license to be designated to the node:

 # mmaddnode -N c6f2bc4n8:quorum::server
mmaddnode: Node c6f2bc4n8.gpfs.net will be designated as possessing server license.
Please confirm that you accept the terms of the IBM Spectrum Scale server Licensing Agreement.
The full text can be found at www.ibm.com/software/sla
Enter "yes" or "no": yes
Tue Mar 12 11:41:51 EDT 2019: mmaddnode: Processing node c6f2bc4n8.gpfs.net
mmaddnode: Command successfully completed
mmaddnode: Propagating the cluster configuration data to all
  affected nodes.  This is an asynchronous process.

The command prompts the user to accept the terms of the licensing agreement.

In the following example the mmaddnode command specifies a license and also specifies the --accept parameter:

 # mmaddnode -N c6f2bc4n8:quorum::server --accept
mmaddnode: Node c6f2bc4n8.gpfs.net will be designated as possessing server license.
Tue Mar 12 11:51:15 EDT 2019: mmaddnode: Processing node c6f2bc4n8.gpfs.net
mmaddnode: Command successfully completed
mmtrace: move /tmp/mmfs/lxtrace.trc.c34f2n03.cpu0 /tmp/mmfs/trcfile.2019-03-12_11.51.27.28959.c34f2n03.cpu0
mmtrace: formatting /tmp/mmfs/trcfile.2019-03-12_11.51.27.28959.c34f2n03 to /tmp/mmfs/trcrpt.2019-03-12_11.51.27.28959.c34f2n03.gz
mmaddnode: Propagating the cluster configuration data to all
  affected nodes.  This is an asynchronous process.

Because the --accept parameter is specified, the command does not need to prompt for acceptance of the license agreement.

In the following example, the mmaddnode command is run with the --on-error-continue option. This option helps to continue the process of adding a list of nodes to the cluster even if certain nodes in the list fail to get added to the cluster.

# mmaddnode -N /tmp/input_nodes.in  --on-error-continue
mmaddnode: On Error Continue option detected

====================================================================
                          Stage 1
Running new node validation and remote initial config...
====================================================================
--------------------------------------------------------------------
PROCESSING NODE: 1
mmaddnode: Node c71f2u33p1.gpfs.net will be designated as possessing server license.
Tue Feb 16 01:31:50 EST 2021: mmaddnode: Processing node c71f2u33p1.gpfs.net
--------------------------------------------------------------------
PROCESSING NODE: 2
mmaddnode: Incorrect node c71f2u33p1.gpfs.net1 specified for command.
--------------------------------------------------------------------
PROCESSING NODE: 3
mmaddnode: Incorrect node c71f2u33p1.gpfs.net2 specified for command.
--------------------------------------------------------------------
PROCESSING NODE: 4
mmaddnode: Incorrect node c71f2u33p1.gpfs.net3 specified for command.
--------------------------------------------------------------------
PROCESSING NODE: 5
mmaddnode: Node c21m2n04.gpfs.net already belongs to the GPFS cluster.
--------------------------------------------------------------------
====================================================================
                        Stage 1 Complete
Number of valid nodes: 1
Total number of nodes: 5
====================================================================

Stage 1 generated files:
      /tmp/mmaddnode_rpt__2021_02_16_01_31_47/valid_nodes_list
      /tmp/mmaddnode_rpt__2021_02_16_01_31_47/invalid_nodes_list

      If stage 2 fails, address any concerns, then rerun the command
      using the 'valid_nodes_list' file with the -N option to avoid
      processing invalid nodes again.
      Rerun the command with the invalid_nodes_list file with the -N
      option once you have addressed the concerns with those invalid
      nodes.

====================================================================
                          Stage 2
Running cluster checks and config repository updates.
This stage has critical checks.  It does not generate valid/invalid
node reports and will exit on any serious errors.
====================================================================
====================================================================
                       Stage 2 Complete
====================================================================
mmaddnode: Command successfully completed
mmaddnode: Propagating the cluster configuration data to all affected nodes.

To add nodes k164n06 as quorum nodes, and make it available as a manager node, issue the following command:

mmaddnode -N k164n06:quorum-manager:k164n06::"Bld 1. RM 205. Row 7b."

To confirm the update and view the comment information, issue the following command:

mmlscluster --comment