Manually change CES HDFS NON-HA cluster into CES HDFS HA cluster

This topic lists the steps to manually change CES HDFS NON-HA cluster into CES HDFS HA cluster.

If the new NameNode is already a part of your IBM Storage Scale cluster, go to the next step. Otherwise, install IBM Storage Scale on that node by following Manually installing the installation toolkit. Then add the new nodes into the existing IBM Storage Scale cluster by running the following command:
```
mmaddnode -N
```
Log in to the new NameNode as root and install the HDFS Transparency package into the new NameNode. Issue the following command on RHEL:
```
# rpm -ivh gpfs.hdfs-protocol-<version>.<arch>.rpm
```
Stop the existing HDFS Transparency NameNode.
```
mmces service stop hdfs
```
Enable CES on the new NameNode by giving the same CES group name as that of the existing HDFS cluster.
```
mmchnode --ces-enable --ces-group hdfscluster -N c16f1n08
```

For the new NameNode, add related property into the hdfs-site.xml.

For example, the existing HDFS NON-HA cluster NameNode is c16f1n07, add another NameNode c16f1n08 to the cluster.

mmhdfs config set hdfs-site.xml -k 
dfs.namenode.shared.edits.dir=file:///gpfs/HA-cluster -k 
dfs.ha.namenodes.cluster=nn1,nn2 -k 
dfs.namenode.rpc-address.cluster.nn2=c16f1n08.gpfs.net:8020 -k 
dfs.namenode.http-address.cluster.nn2=c16f1n08.gpfs.net:50070

Upload the configuration into CCR.
```
mmhdfs config upload
```
Initialize the NameNode shared directory to store HDFS cluster HA info from one NameNode.
```
/usr/lpp/mmfs/hadoop/bin/hdfs namenode -initializeSharedEdits
```

Start the existing HDFS Transparency NameNode.

mmces service start hdfs -N c16f1n07,c16f1n08

mmces service start hdfs -a

If the CES HDFS cluster is Kerberos enabled, ensure that you configure Kerberos for the new NameNode by following Setting up Kerberos for HDFS Transparency nodes.
Check the status of the added NameNodes in the HDFS cluster.
```
mmces service list -a
```
Check the status of both the NameNodes. One should be Active and the other should be in standby.
```
/usr/lpp/mmfs/hadoop/bin/hdfs haadmin -getAllServiceState
```
Restart DataNodes to take effect.
```
mmhdfs hdfs-dn restart
```
Check the status of the DataNodes.
```
mmhdfs hdfs-dn status
```