Enabling HDFS encryption

Edit online

This topic lists the steps to enable HDFS encryption for CES HDFS and CDP Private Cloud Base clusters.

Before you begin

Note:
  • Enabling HDFS encryption for CES HDFS and CDP Private Cloud Base clusters requires that the Ranger, TLS and Kerberos are enabled. Before you proceed to enable HDFS encryption, ensure that Ranger, TLS and Kerberos are fully functional.
  • Ensure Ranger policies are working properly by following the steps in Verifying Ranger policy. Otherwise Ranger KMS server might fail to start.
  • HDFS encryption requires the Ranger-KMS with Key Trustee Server Cloudera service and the Key Trustee Server (KTS) service. These two services can be added only after the CDP Private Cloud Base cluster is deployed and not during the first time the cluster is created.

Procedure

  1. Configure Key Trustee Server (KTS) repository in Cloudera Manager.
    1. Go to Cloudera Manager > Parcels > Parcel Repositories & Network Settings.
    2. Add Repository URL where KEYTRUSTEE_SERVER-X.X.X.X-X.keytrusteeX.X.X.X.pX.XXXXXXX-XXX.parcel is present in Remote Parcel Repository URLs.
    3. Click Save & Verify Configuration.
    4. Click Close.
    5. On KEYTRUSTEE_SERVER, click Download > Distribute > Activate.
  2. Add Key Trustee Server (KTS) service to CDP cluster.
    1. Go to Cloudera Manager, click Cluster > Actions > Add Service and install the Key Trustee Server (KTS) service. Follow the wizard and install the service.
      Note: While on the Setup Entropy screen, if sufficient entropy is available, you can skip the installation of rng-tools. Otherwise, if entropy drops while generating the secrets, the Cloudera Manager wizard might become unresponsive. In that case, go back to the Setup Entropy screen to install and configure the rng-tools package.
    2. Start Key Trustee Server from the Cloudera Manager UI.
  3. Add Ranger KMS with Key Trustee Server to CDP cluster.
    1. Go to Cloudera Manager, click Cluster > Actions > Add Service and install the Ranger KMS with Key Trustee Server service.
    2. Follow the wizard and complete the installation.
  4. Update CES HDFS configuration to enable native HDFS encryption.
    1. Stop the IBM Storage® Scale service from Cloudera Manager. The NameNodes and DataNodes should be stopped.
    2. Log into a CES HDFS node and run the following commands to update the CES HDFS configuration to enable encryption:
      # mmhdfs config set gpfs-site.xml -k gpfs.encryption.enabled=true -k gpfs.ranger.enabled=scale
# mmhdfs config set core-site.xml -k hadoop.security.key.provider.path=kms://https@<RANGER_KMS_HOST>:9494/kms
      Replace <RANGER_KMS_HOST> with the FQDN hostname of your Ranger KMS server.
      Note: If you had moved your Ranger KMS service from one host to another, ensure that you update the hadoop.security.key.provider.path parameter to the correct host.
    3. Upload the configuration to CCR
      # mmhdfs config upload
    4. Go to Cloudera Manager, click Cluster > Actions menu.
    5. Stop all the services.
    6. Deploy client configuration.
    7. Start all the services.
    Note: