Enabling HDFS and Solr encryption
Procedure
- Log in to the Ranger KMS console:
https://<ranger_server>:6182/index.html
The default credentials are keyadmin/keyadmin.
Tip: You can find the Ranger server details by logging into the Ambari console, and selecting Ranger > Configs > Advanced. Use the Enternal URL value under Ranger Setting. - Create an encryption key that is named
sifshdfskey.
- In the Ranger console, click the Encryption tab.
- Select the KMS Service.
- Click Add New Key.
- Enter sifshdfskey as the key name.
- Select the cipher name. Ranger supports AES/CTR/NoPadding as the cipher suite.
- Specify the key length as 128 bits.
- Add other attributes as needed, and save the key.
- Create an encryption key that is named solrhdfskey.
- In the Ranger console, click the Encryption tab.
- Select the KMS Service.
- Click Add New Key.
- Enter solrhdfskey as the key name.
- Select the cipher name. Ranger supports AES/CTR/NoPadding as the cipher suite.
- Specify the key length as 128 bits.
- Add other attributes as needed, and save the key.
- Create an encryption zone on one of the HDFS nodes.
- Log in to an HDFS node computer as the hdfs user.
- Run the following commands:
hdfs crypto -createZone -keyName sifshdfskey -path /user/sifsuserhdfs dfs -chown sifsuser:hadoop /user/sifsuserhdfs crypto -createZone -keyName solrhdfskey -path /user/solruserhdfs dfs -chown solruser:hadoop /user/solruser - To verify the zone, run the following command:
hdfs crypto -listZonesThe new zone and its associated key should be listed.
$ hdfs crypto -listZones /user/sifsuser sifshdfskey /user/solruser solrhdfskey
- Create access policies for the encryption keys.
- Log in to the Ranger console as the keyadmin user.
- In Service Manager, click fcicluster_kms, and click Add New Policy.
- Create a policy that is named sifspolicy, and set the following
values:
- In Key Name, enter sifshdfskey.
- In Select User, enter sifsuser, hdfs.
- In Permissions, click Add, and select Decrypt_EEK, and click the check mark to add the permission.
- In Delegate Admin, select yes.
- Click Add.
- Create a policy that is named solrpolicy, and set the following
values:
- In Key Name, enter solrhdfskey.
- In Select User, enter solruser.
- In Permissions, click Add, and select Decrypt_EEK, and click the check mark to add the permission.
- In Delegate Admin, select yes.
- Click Add.
- Verify that the contents are encrypted.
- Log in to one of the HDFS nodes as the sifsuser user.
- Enter the following commands to write a test file that is named
testdata.txt onto HDFS:
echo "Verification of encryption" > testdata.txthdfs dfs -put testdata.txt /user/sifsuser/ - Enter the following command:
hdfs dfs -cat /user/sifsuser/testdata.txtThis should show clear text data.
- Log into the HDFS master node as the hdfs user, and run the
following command:
hdfs dfs -cat /.reserved/raw/user/sifsuser/testdata.txtThis should show encrypted data.
Note: If the Kerberos session has expired, run the following command as sifsuser:kinit –kt /etc/security/keytabs/sifsuser.keytab sifsuser@IBM.COM