Setup the HDFS Transparency cluster

Setting up the HDFS Transparency for CDP Private Cloud Base using CES HDFS protocol cluster setup.

Figure 1. CES HDFS cluster installation overview
CES HDFS cluster installation overview
Prerequisite:
  • Ensure that the shared storage is set up and available.
Steps:
  1. Configure IBM Storage Scale on the CES HDFS cluster to access the shared storage. For example, the IBM Storage Scale cluster accesses IBM Storage Scale System through the remote mount. After the storage is accessible, create the CES HDFS NameNodes and DataNodes onto the CES HDFS cluster. For information about setting up and installing CES HDFS, see Installing.
  2. Optional: Enable Kerberos. To enable Kerberos, see Enabling Kerberos for CES HDFS and CDP Private Cloud Base.
  3. Optional: Enable TLS. To enable TLS, see Enable TLS.
  4. Set up the users and groups in the CES HDFS cluster. All the users and groups id must be same in an IBM Storage Scale cluster to work properly. To set up the users and groups id onto the IBM Storage Scale CES HDFS cluster, see Configuring users, groups and file system access for IBM Storage Scale.
  5. Verify the CES HDFS cluster to ensure you can read and write to the storage file system.

After the CES HDFS cluster is configured, you can set up the CDP Private Cloud cluster.

Note:
  • When IBM Storage Scale is installed, the IBM Storage Scale CSD (gpfs.hdfs.cloudera.cdp.csd-<version-number>.noarch.rpm) package resides in /usr/lpp/mmfs/<IBM Storage Scale version>/hdfs_rpms/rhel/hdfs_3.1.1.x. You need to move the package to the CDP Private Cloud cluster.
  • If on the CES HDFS cluster NameNode HA, Kerberos and/or TLS are enabled, set up the CDP Private Base cluster with the same configuration.