Enabling HDFS and Solr encryption

Procedure

  1. Log in to the Ranger KMS console:

    https://<ranger_server>:6182/index.html

    The default credentials are keyadmin/keyadmin.

    Tip: You can find the Ranger server details by logging into the Ambari console, and selecting Ranger > Configs > Advanced. Use the Enternal URL value under Ranger Setting.
  2. Create an encryption key that is named sifshdfskey.
    1. In the Ranger console, click the Encryption tab.
    2. Select the KMS Service.
    3. Click Add New Key.
    4. Enter sifshdfskey as the key name.
    5. Select the cipher name. Ranger supports AES/CTR/NoPadding as the cipher suite.
    6. Specify the key length as 128 bits.
    7. Add other attributes as needed, and save the key.
  3. Create an encryption key that is named solrhdfskey.
    1. In the Ranger console, click the Encryption tab.
    2. Select the KMS Service.
    3. Click Add New Key.
    4. Enter solrhdfskey as the key name.
    5. Select the cipher name. Ranger supports AES/CTR/NoPadding as the cipher suite.
    6. Specify the key length as 128 bits.
    7. Add other attributes as needed, and save the key.
  4. Create an encryption zone on one of the HDFS nodes.
    1. Log in to an HDFS node computer as the hdfs user.
    2. Run the following commands:
      hdfs crypto -createZone -keyName sifshdfskey -path /user/sifsuser
      hdfs dfs -chown sifsuser:hadoop /user/sifsuser
      hdfs crypto -createZone -keyName solrhdfskey -path /user/solruser
      hdfs dfs -chown solruser:hadoop /user/solruser
    3. To verify the zone, run the following command:
      hdfs crypto -listZones

      The new zone and its associated key should be listed.

      $ hdfs crypto -listZones
      /user/sifsuser  sifshdfskey
      /user/solruser  solrhdfskey
  5. Create access policies for the encryption keys.
    1. Log in to the Ranger console as the keyadmin user.
    2. In Service Manager, click fcicluster_kms, and click Add New Policy.
    3. Create a policy that is named sifspolicy, and set the following values:
      • In Key Name, enter sifshdfskey.
      • In Select User, enter sifsuser, hdfs.
      • In Permissions, click Add, and select Decrypt_EEK, and click the check mark to add the permission.
      • In Delegate Admin, select yes.
    4. Click Add.
    5. Create a policy that is named solrpolicy, and set the following values:
      • In Key Name, enter solrhdfskey.
      • In Select User, enter solruser.
      • In Permissions, click Add, and select Decrypt_EEK, and click the check mark to add the permission.
      • In Delegate Admin, select yes.
    6. Click Add.
  6. Verify that the contents are encrypted.
    1. Log in to one of the HDFS nodes as the sifsuser user.
    2. Enter the following commands to write a test file that is named testdata.txt onto HDFS:
      echo "Verification of encryption" > testdata.txt
      hdfs dfs -put testdata.txt /user/sifsuser/
    3. Enter the following command:
      hdfs dfs -cat /user/sifsuser/testdata.txt

      This should show clear text data.

    4. Log into the HDFS master node as the hdfs user, and run the following command:
      hdfs dfs -cat /.reserved/raw/user/sifsuser/testdata.txt

      This should show encrypted data.

      Note: If the Kerberos session has expired, run the following command as sifsuser:
      kinit –kt /etc/security/keytabs/sifsuser.keytab sifsuser@IBM.COM