Multiple IBM Storage Scale file system support
Set up a CES HDFS cluster with storage type set to shared,shared. The
storage setup must consist of two file
system.
To add support for multiple IBM StorageĀ® Scale file
system, follow the steps listed below:
- Set up the two file systems.
- Set up CES HDFS cluster using the first mounted file system for HDFS and cesSharedRoot.
- Verify the NameNode state and the health of CES HDFS nodes using the first mounted file system.
- Add the second mounted file system using the following steps:
- Stop all CES HDFS services in the CES HDFS cluster by running the following
command:
mmhdfs hdfs stop - Edit the values of the gpfs.storage.type, gpfs.mnt.dir
and gpfs.replica.enforced parameters in
/var/mmfs/hadoop/etc/hadoop/gpfs-site.xml as
follows:
<property> <name>gpfs.mnt.dir</name> <value>/fs1,/fs2</value> </property> <property> <name>gpfs.storage.type</name> <value>shared,shared</value> </property> <property> <name>gpfs.replica.enforced</name> <value>gpfs,gpfs</value> </property>
- Stop all CES HDFS services in the CES HDFS cluster by running the following
command:
- Upload the config to update the CCR by running the following
command:
mmhdfs config upload - Start the CES HDFS services and HDFS Transparency by running the following
command:
mmhdfs hdfs start - Verify the state of the NameNode and DataNode by running the following
commands:
mmhdfs hdfs-nn status mmhdfs hdfs-dn statusNote: To list all the files/directories, run the following command:hadoop dfs -ls /