Enabling Ranger

This topic lists the steps to enable Ranger.

Prerequisites
  • Ensure that the CES HDFS cluster is functional. If the IBM Storage Scale service is already added to Cloudera Manager, ensure that the CDP Private Cloud Base cluster that is integrated with the IBM Storage Scale service is functional as well. For verifying, follow the steps listed in Verifying installation.
  • Before you enable Ranger, Kerberos must be enabled on CES HDFS and the CDP Private Cloud Base clusters.
  • All the CDP Private Cloud Base services should have Kerberos enabled. Click Administration > Security > Status tab > Kerberos > Enabled check-box. You must see all the services enabled for Kerberos. For more information, see Problem determination.
  • Before creating the database for Ranger, ensure that you perform a workaround for MySQL/MariaDB by following Installing Ranger service may fail with the following SQL error from MySQL/MariaDB.

It is recommended to add the Solr and Ranger services together with the IBM Storage Scale service at the time of initial CDP Private Cloud Base cluster creation. However, you can add these services later as well.

Overall flow required to enable Ranger is as follows:

  1. If the CDP Private Cloud Base cluster with IBM Storage Scale already exists, then stop all the cluster services.
  2. Add the Solr and Ranger services in Cloudera Manager.
  3. Configure the IBM Storage Scale service for Ranger.
  4. Configure the CES HDFS cluster for Ranger.
  5. Start all CDP Private Cloud Base cluster services.
Procedure
  1. Stop all the CDP Private Cloud Base cluster services from Cloudera Manager.

    If the HDFS Transparency services were started manually using the mmhdfs/mmces commands, stop them as well.

  2. Add the Solr and Ranger services.
    If Solr is being added to an existing CDP Private Cloud Base cluster with IBM Storage Scale, then the following workaround is needed for Solr to start properly:
    • In Cloudera Manager console, click Solr > Configuration, search for ZNode and set the value of the Solr configuration parameter ZooKeeper ZNode to /solr-infra.
    • Ensure that the Kerberos configuration checkbox is enabled for the Solr and Ranger services.
    • After adding Ranger, the Solr service changes its name to CDP-INFRA-SOLR.
  3. Configure the IBM Storage Scale service for Ranger.
    1. Click IBM Spectrum Scale > Configuration. Search for hadoop.security.authorization and enable this option by clicking on the checkbox.
    2. Go to Cloudera Manager, click IBM Spectrum Scale > Configuration > Transparency NameNode Advanced Configuration Snippet (Safety Valve) for ranger-hdfs-security.xml, and add the following custom configuration:
      Name: ranger.plugin.hdfs.policy.rest.ssl.config.file
Value: ranger-hdfs-policymgr-ssl.xml
    3. Click IBM Spectrum Scale > Configuration > Transparency NameNode Advanced Configuration Snippet (Safety Valve) for ranger-hdfs-policymgr-ssl.xml and add the following custom configurations:
              <property>
            <name>xasecure.policymgr.clientssl.truststore</name>
            <value>/var/lib/cloudera-scm-agent/agent-cert/spectrum_scale_ces_hdfs_truststore.jks</value>
        </property>
        <property>
            <name>xasecure.policymgr.clientssl.truststore.type</name>
            <value>jks</value>
        </property>
        <property>
            <name>xasecure.policymgr.clientssl.truststore.credential.file</name>
            <value>jceks://file/var/lib/cloudera-scm-agent/agent-cert/rangerpluginssl.jceks</value>
        </property>
        <property>
            <name>hadoop.security.credential.provider.path</name>
            <value>jceks://file/var/lib/cloudera-scm-agent/agent-cert/rangerpluginssl.jceks</value>
        </property>
    4. If you are using CDP Private Cloud Base 7.1.8 or later, in Cloudera Manager navigate to Core Configuration Service > Configuration and search for Additional Rules to Map Kerberos Principals to Short Names.

      If you are using the earlier versions of CDP Private Cloud Base, in Cloudera Manager navigate to IBM Spectrum Scale > Configuration and search for Additional Rules to Map Kerberos Principals to Short Names.

      Then, change the {DEFAULT_RULES} value to the following set of rules as needed for Ranger and HDFS Transparency:

      RULE:[2:$1@$0](rangeradmin@IBM.COM)s/(.*)@IBM.COM/ranger/
RULE:[2:$1@$0](rangertagsync@IBM.COM)s/(.*)@IBM.COM/rangertagsync/
RULE:[2:$1@$0](rangerusersync@IBM.COM)s/(.*)@IBM.COM/rangerusersync/
RULE:[2:$1@$0](rangerkms@IBM.COM)s/(.*)@IBM.COM/keyadmin/
RULE:[2:$1/$2@$0](nn/.*@.*IBM.COM)s/.*/hdfs/
RULE:[2:$1/$2@$0](dn/.*@.*IBM.COM)s/.*/hdfs/
RULE:[1:$1@$0](hdfs@IBM.COM)s/@.*//
RULE:[1:$1@$0](.*@IBM.COM)s/@.*//
      Note: Additional Rules to Map Kerberos Principals to Short Names is translated to auth_to_local rules in core-site.xml. Cloudera Manager appends some more rules followed by 'DEFAULT' as the last line. Do not add 'DEFAULT' as the last line in this text field because it prevents the translation of the rules appended by Cloudera Manager.

      Replace IBM.COM with your Kerberos Realm name in the above example rules.

    5. Save and deploy the client configuration. Do not start any services yet.
  4. Configure HDFS Transparency for Ranger.
    1. Enabling Ranger requires setting the server-side configuration on the HDFS Transparency NameNodes.
      1. Update the HDFS Transparency configuration files and upload the changes.
        • Log in to one of the CES HDFS NameNode.
        • Get the configuration files by running the following commands: 
          # mkdir /tmp/hdfsconf
# mmhdfs config export /tmp/hdfsconf core-site.xml,hdfs-site.xml,hadoop-env.sh,gpfs-site.xml
# cd /tmp/hdfsconf/
        • Update the config files in /tmp/hdfsconf with the following changes based on your environment:
          File: core-site.xml
          <property>
<name>hadoop.security.auth_to_local</name>
<value>
RULE:[2:$1@$0](rangeradmin@IBM.COM)s/(.*)@IBM.COM/ranger/
RULE:[2:$1@$0](rangertagsync@IBM.COM)s/(.*)@IBM.COM/rangertagsync/
RULE:[2:$1@$0](rangerusersync@IBM.COM)s/(.*)@IBM.COM/rangerusersync/
RULE:[2:$1@$0](rangerkms@IBM.COM)s/(.*)@IBM.COM/keyadmin/
….. <other existing rules>
DEFAULT
</value>
</property>
          File: hdfs-site.xml
            <property>  
    <name>dfs.permissions</name>
    <value>true</value>
  </property>
  <property>
    <name>dfs.permissions.enabled</name>
    <value>true</value>
  </property>
  <property>
    <name>dfs.permissions.ContentSummary.subAccess</name>
    <value>true</value>
  </property>
 <property>
    <name>dfs.namenode.inode.attributes.provider.class</name>
    <value>org.apache.ranger.authorization.hadoop.RangerHdfsAuthorizer</value>
  </property>
          File: gpfs-site.xml
          <property>
    <name>gpfs.ranger.enabled</name>
    <value>scale</value>
  </property>
          File: hadoop-env.sh
          Note: Based on your environment, substitute the right path to the CDH ranger-hdfs-plugin library.
          for f in /opt/cloudera/parcels/CDH/lib/ranger-hdfs-plugin/lib/*.jar;
do
export HADOOP_CLASSPATH=$HADOOP_CLASSPATH:$f
done

for f in /opt/cloudera/parcels/CDH/lib/hadoop/client/jersey-client.jar;
do
export HADOOP_CLASSPATH=$HADOOP_CLASSPATH:$f
done
export HADOOP_CLASSPATH=$HADOOP_CLASSPATH: /opt/cloudera/parcels/CDH-7.1.7-1.cdh7.1.7.p0.15945976/lib/hadoop/client/jackson-core-asl.jar
          When Auto-TLS is enabled in Cloudera Manager and either of the following conditions are true, additional configuration is needed:
          • Ranger service is enabled after the creation of the initial IBM Storage Scale integrated CDP cluster
          • auto-TLS is enabled in Cloudera manager after the creation of the initial IBM Storage Scale integrated CDP cluster
          • Ranger High Availability (HA) is enabled
          File: hadoop-env.sh
          export HADOOP_CREDSTORE_PASSWORD=none
      2. Import the files into CES HDFS cluster by running the following command:
        # mmhdfs config import /tmp/hdfsconf  core-site.xml
# mmhdfs config import /tmp/hdfsconf  hdfs-site.xml
# mmhdfs config import /tmp/hdfsconf  hadoop-env.sh
      3. Upload the changes to CES HDFS cluster by running the following command.
        # mmhdfs config upload
      4. Additional configurations when TLS is enabled:
        • Ranger needs the IBM Storage Scale CES HDFS Truststore password to be encrypted in a jceks file. On a CES NameNode, create the jceks file for Ranger with the following command:
          # java -cp "/opt/cloudera/parcels/CDH/lib/ranger-hdfs-plugin/install/lib/*" org.apache.ranger.credentialapi.buildks create "sslTrustStore" -value <truststore_password> -provider "jceks://file/var/lib/cloudera-scm-agent/agent-cert/rangerpluginssl.jceks" -storetype "jceks"

          Replace <truststore_password> with the corresponding actual passwords from under Manually enabling TLS for HDFS Transparency, or if the automation script was used then the same can be retrieved from the /var/mmfs/hadoop/etc/hadoop/ssl-server.xml file.

        • Validate the above jceks file (optional):
          # HADOOP_CREDSTORE_PASSWORD=none java -cp /opt/cloudera/cm/lib/security-*.jar com.cloudera.enterprise.crypto.GenericKeyStoreTypePasswordExtractor "jceks" "/var/lib/cloudera-scm-agent/agent-cert/rangerpluginssl.jceks" "sslTrustStore"
        • Distribute the above jceks file to all the CES HDFS NameNodes.
          For each CES HDFS, run the following command:
          scp /var/lib/cloudera-scm-agent/agent-cert/rangerpluginssl.jceks root@<CES HDFS Host>:/var/lib/cloudera-scm-agent/agent-cert/
      5. Ensure that you create ranger, rangertagsync, rangerusersync and keyadmin user using the gpfs_create_hadoop_users_dirs.py script. Log in to a CES HDFS NameNode and run the following commands:
        /usr/lpp/mmfs/hadoop/scripts/gpfs_create_hadoop_users_dirs.py --create-custom-hadoop-user-group ranger
/usr/lpp/mmfs/hadoop/scripts/gpfs_create_hadoop_users_dirs.py --create-custom-hadoop-user-group rangertagsync
/usr/lpp/mmfs/hadoop/scripts/gpfs_create_hadoop_users_dirs.py --create-custom-hadoop-user-group rangerusersync
/usr/lpp/mmfs/hadoop/scripts/gpfs_create_hadoop_users_dirs.py --create-custom-hadoop-user-group keyadmin
  5. Start services:
    • If TLS is enabled, start the NameNodes using the workaround mentioned in issue and then start all the services from Cloudera Manager.
    • If TLS is not enabled, start all the services from Cloudera Manager as usual.