Manually adding a new HDFS cluster into existing HDFS cluster on the same GPFS cluster (Multiple HDFS clusters)

This topic lists the steps to manually add a new HDFS cluster into existing HDFS cluster on the same GPFS cluster (Multiple HDFS clusters).

1. Create different CES groups for different HDFS clusters and ensure that the existing HDFS cluster nodes are different than the new HDFS cluster nodes that will be added.
2. For the first HDFS cluster, HDFS configuration including core-site.xml, hdfs-site.xml, gpfs-site.xml and hadoop-env.sh must be executed on the NameNode belonging to that cluster. Run the following command to upload the configuration into CCR:

   /usr/lpp/mmfs/hadoop/sbin/mmhdfs config upload

3. Start the NameNodes service of the first HDFS cluster.

   mmces service start hdfs -a

4. Check that the NameNodes service status is running for the first HDFS cluster.

   mmces service list -a

5. For the second HDFS cluster, HDFS configuration including core-site.xml, hdfs-site.xml, gpfs-site.xml and hadoop-env.sh must be executed on the NameNode belonging to the second cluster.
   The value of dfs.nameservices should be set to the cluster name of the second HDFS cluster. Run the following command to upload the configuration into CCR. The configuration uploaded will be pushed to all the NameNodes of the new HDFS cluster.

   /usr/lpp/mmfs/hadoop/sbin/mmhdfs config upload

6. Enable CES for the NameNodes of the new HDFS cluster and set the CES group name to the CES group name of the new HDFS cluster.

   mmchnode --ces-enable --ces-group=[groupname_addedhdfscluster] -N NewClusterNameNode1,NewClusterNameNode2

7. Start the NameNodes service of the new HDFS cluster, if not started already.

   mmces service start hdfs -a

8. Check if the NameNodes service status is running for the new HDFS cluster.

   mmces service list -a

9. Log in to one of the newly added NameNodes and start the DataNodes of the new HDFS cluster.

   mmhdfs hdfs-dn start