Configuring Kerberos using the Kerberos script provided with IBM Storage Scale

From HDFS Transparency 3.1.1-3, IBM Storage Scale provides a Kerberos configuration script /usr/lpp/mmfs/hadoop/scripts/gpfs_kerberos_configuration.py to help with setting up Kerberos for HDFS Transparency interactively.

From HDFS Transparency 3.1.1-4, a non-interactive version of the automation script is also supported. The input parameters can be specified through a customized json input file.

The output of the script is logged to /var/log/kerberos_configuration_setup.log file.
Note: If you need to set up more than one HDFS Transparency cluster using a common KDC server , see the Limitation in the Kerberos topic.

Before following these steps, see the Prerequisites topic.

There are two methods to use the Kerberos script:
  1. Interactive method
  2. Custom json file method

Interactive method

You can perform the following using the interactive method:
  1. Set up a new KDC server. If you already have a KDC server, go to step 2.
    Setting up a new KDC server helps with the following:
    1. Install and configure a new Kerberos server on the host being run. Create or update the /var/kerberos/krb5kdc/kdc.conf and /etc/krb5.conf files.
    2. By default, the principals are configured such that ticket_lifetime is set to 24h and renew_lifetime is set to 7d. If needed, update these default values.
  2. Configure Kerberos for HDFS Transparency.

    Configuring Kerberos helps with the following:

    1. Install and configure Kerberos client on the HDFS Transparency nodes.
    2. Create host principals.
    3. Create NameNode and DataNode principals and keytabs for HDFS Transparency.
    4. Create hdfs user principal and keytab.
    5. Apply the Kerberos configurations for hdfs-site.xml, core-site.xml and hadoop-env.sh for HDFS Transparency.
  3. Clear Kerberos configuration from HDFS Transparency.
    Clearing Kerberos configuration helps with the following:
    1. Disable the Kerberos configurations from HDFS Transparency.
    2. In case you want to re-enable Kerberos at a later time, the existing principals and keytabs created for NameNodes and DataNodes are retained.
Perform the following to run the gpfs_kerberos_configuration.py script:
  • For HDFS Transparency-3.1.1-3:
    # /usr/lpp/mmfs/hadoop/scripts/gpfs_kerberos_configuration.py
      MIT Kerberos configuration:
      1: Setup a new KDC server.
           [Run the script on the KDC server host]
      2: Configure Kerberos for HDFS Transparency.
           [Run the script on a CES-HDFS cluster node that has password-less SSH access to the other HDFS Transparency nodes]
      3: Clear Kerberos configuration from HDFS Transparency.
           [This option will remove the Kerberos configurations from your HDFS Transparency cluster. 
            This will not remove the existing principals and keytabs for NameNodes and DataNodes] 

      Choose option 1/2/3:
  • For HDFS Transparency-3.1.1-4:
    # /usr/lpp/mmfs/hadoop/scripts/gpfs_kerberos_configuration.py
      MIT Kerberos configuration:
      1: Setup a new KDC server.
           [Run the script on the KDC server host]
      2: Configure Kerberos for HDFS Transparency.
           [Run the script on a CES-HDFS cluster node that has password-less SSH access to the other HDFS Transparency nodes]
      3: Clear Kerberos configuration from HDFS Transparency.
           [This option will remove the Kerberos configurations from your HDFS Transparency cluster. 
            This will not remove the existing principals and keytabs for NameNodes and DataNodes]
      4: Exit.

      Choose option 1/2/3/4:

Custom json file method

For this method, the user needs to update the custom json file (/usr/lpp/mmfs/hadoop/scripts/gpfs_kerberos_config_metadata.json) with inputs specific to the environment. Then run the gpfs_kerberos_configuration.py script as follows:
[root@scripts]# ./gpfs_kerberos_configuration.py -h
usage: gpfs_kerberos_configuration.py [-h] [-c CONFIG]Create Kerberos configurationoptional arguments:
  -h, --help            show this help message and exit
  -c CONFIG, --config CONFIG
                        Provide 'gpfs_kerberos_config_metadata.json' config
                        path. Help: The sample config template file can be
                        found in '/usr/lpp/mmfs/hadoop/scripts/gpfs_kerberos_c
                        onfig_metadata.json'Example:
[root@scripts]#./gpfs_kerberos_configuration.py -c /usr/lpp/mmfs/hadoop/scripts/gpfs_kerberos_config_metadata.json