Configuring

The following configurations are for manually configuring HDFS Transparency. For example, set up HDFS Transparency for open-source Apache or Cloudera CDP stack.

For configuring, Hadoop distribution must be installed under $YOUR_HADOOP_PREFIX on each machine in the Hadoop cluster. The configurations for IBM Storage® Scale HDFS transparency are located under /usr/lpp/mmfs/hadoop/etc/hadoop (for HDFS Transparency 2.7.3-x) or /var/mmfs/hadoop/etc/hadoop (for HDFS Transparency 3.0.x) for any Hadoop distribution. Configuration files for Hadoop distribution are located in different locations. For example, /etc/hadoop/conf for Cloudera CDP.

The core-site.xml and hdfs-site.xml configuration files should be synchronized between all the nodes and kept identical for the IBM Storage Scale HDFS Transparency and Hadoop Distribution. The log4j.properties configuration file can differ between the IBM Storage Scale HDFS Transparency and the open-source Apache Hadoop distribution.