Adding a new HDFS cluster into existing HDFS cluster on the same GPFS cluster using install toolkit

The Using installation toolkit section describes how to add in a new HDFS cluster into the environment.

The difference when creating another HDFS cluster into an existing HDFS cluster on the same GPFS cluster is to create a different cluster name for the new HDFS cluster.

For example, use CLUSTER2 as the cluster name for the second HDFS cluster to be added into the existing 1st HDFS cluster:
  1. Add the new 2nd HDFS cluster nodes into the GPFS cluster.
    Ensure that the nodes are new nodes and not a part of the existing HDFS cluster.
    # NameNodes (Protocol node)
./spectrumscale node add c902f09x01.gpfs.net -p
./spectrumscale node add c902f09x02.gpfs.net -p

# DataNodes
./spectrumscale node add c902f09x03.gpfs.net
./spectrumscale node add c902f09x04.gpfs.net
./spectrumscale node add c902f09x05.gpfs.net
./spectrumscale node add c902f09x06.gpfs.net
  2. Configure the 2nd cluster CES HDFS cluster.
    ./spectrumscale config hdfs new -n CLUSTER2 -nn NAMENODES -dn DATANODES -f FILESYSTEM -d DATADIR
./spectrumscale config hdfs new -n CLUSTER2 -nn c902f09x01.gpfs.net, c902f09x01.gpfs.net -dn 
c902f09x03.gpfs.net, c902f09x04.gpfs.net, c902f09x05.gpfs.net, c902f09x06.gpfs.net -f gpfs -d gpfshdfs2
  3. Deploy the 2nd cluster.
    ./spectrumscale deploy -pr 
./spectrumscale deploy
    Note:
    • Ensure that there are sufficient free CES-IPs available for usage.
    • Ensure that the new cluster NameNodes and DataNodes are not the same nodes as the existing HDFS cluster.
    • Ensure that the DATADIR is unique to host the second cluster’s data.