Introduction

In a previous post, it demonstrated how to configure LDAP integration with IBM Open Platform on a BigInsights Cluster. Here is the link.
In this post, it concentrates on the missing content from the previous post for the Microsoft Active Directory (AD) integration.

Objective

This technical document is intended to show viewers step by step instructions on how to setup AD on RedHat and/or CentOS Operating System and the integration on a Hadoop Cluster.

Version Tested

• RedHat v7.x, CentOS v7.x
• Ambari v2.4.x

Lessons Learned

• Don’t include spaces in a bind id (ex. “Service\, EnterpriseLdap”)

AD Configuration Steps

1. Collect the Microsoft Active Directory (AD) server information

• AD server FQDN. Example: ldap2.ibm.com
• AD server port. Default is 389.
• AD server BaseDN. (Currently it does not support case sensitive AD entries) Ex. dc=ibm,dc=com
• AD server Admin account and its credential. Ex. cn=Manager,ou=IT,ou=Departments,dc=ibm,dc=com

2. Install OpenLDAP pre-requisites on all Hadoop cluster nodes

• Run the command: $sudo yum -y install openldap openldap-clients nss-pam-ldapd pam

3. Setup the configuration files

• Modify file “/etc/nslcd.conf” with following:

uri ldap://ldap2.ibm.com:389;
base dc=us,dc=ibm,dc=com
binddn cn=manager,OU=IT,OU=Departments,DC=us,DC=ibm,DC=com
bindpw <yoursecretpassword>
pagesize 1000
referrals off
idle_timelimit 800
filter passwd (&(objectClass=user)(!(objectClass=machines)))
map passwd uid sAMAccountName
map passwd uidNumber objectSid:
map passwd gidNumber primaryGroupID
map passwd homeDirectory “/home/$sAMAccountName”
map passwd loginShell “/bin/bash”
filter shadow (&(objectClass=user)(!(objectClass=machine)))
map shadow uid sAMAccountName
map shadow shadowLastChange pwdLastSet
filter group (objectClass=group)
map group gidNumber objectSid:
ssl no
tls_cacertdir /etc/openldap/cacerts

• Modify file “/etc/nsswitch.conf” with following:

passwd: files sss ldap
shadow: files sss ldap
group: files sss ldap
hosts: files dns
netgroup: files sss ldap
automount: files ldap

• Modify file “/etc/openldap/ldap.conf” with following:

TLS_CACERTDIR /etc/openldap/cacerts
SASL_NOCANON on
URI ldap://ldap2.ibm.com:389
BASE dc=ibm,dc=com

4. (Re-)Start nslcd Service

$ sudo systemctl restart nslcd

5. Test AD client install and connectivity to the server

• Login to Ambari node as root
• Run connectivity test: Ex. $ sudo openssl s_client -connect ad.us.ibm.com:636
  Note: If connectivity test failed, you should verify if AD server’s firewall is opened for port 389 and 636 (Ex. $ sudo netstat -anp|grep 636)
• Test if a AD user can login to Ambari node without an existing local Unix account
  1. $ ssh aduser1@us.ibm.com
  2. You may get an error message indicating that command failed to change home directory for ad user. This is a correct behavior since ad user does not exist on the local machine.
  3. If you get the following status, you have successfully logged in using AD user id and it’s credential. You are done setting up AD client and successfully tested connections. Now you are ready to move on to configure Ambari LDAP.

     

6. Setup Ambari AD configuration

1. Login to Ambari node as user “root”
2. Either using CLI or direct modify Ambari property file for advanced users. Here is showing via CLI commands.
   • $ sudo ambari-server setup-ldap
   • Answer each question from the prompt. Notice I chose anonymous bind as false so no users can login anonymously.
   • Once you save the settings, the CLI program saves the parameter values into file “/etc/ambari-server/conf/ambari.properties”

     

3. Restart Ambari process. Ex. $ sudo ambari-server restart
4. Sync up AD users into Ambari metadata store so that Ambari Web UI can allow AD users to login.
   ** Be careful of this step. In the example is showing sync all users. You do not want to do this in a production large enterprise environment, you need to narrow it down to a selected targeted users or groups. Ex. $ sudo ambari-server sync-ldap –all (notice there is a double dashes before word “all”)

   

7. AD User Test in Ambari

Now let’s test Ambari Web UI login as a AD user. (Screen shot below). Notice that Ambari Admin user needs to assign appropriate permission to each ad user before s/he can see any screens when they logging in. If not, s/he still can login but will see a blank page.Screen shot #1:Notice all ad users showed up after sync command. None of those users exist on current cluster nodes nor do you need to create them on cluster servers.



Screen shot #2:Ambari administrator must assign permissions to the AD users.



Screen shot #3:When Ambari Admin assigned AD users operator permission. S/He can then login and act as a Hadoop admin to monitoring and maintaining the cluster.

AD / Knox Setup Steps

1. Login to Ambari Web UI as administrator. (ie. “http://<ambari-server>:8080” default port is 8080. You can change the port in ambari.properties file)
2. Click on “Knox” from left panel. Select “Configs” tab from right panel.
3. Extend the section “Advanced topology”, and modify the following entries:
   • main.ldapRealm.userDnTemplate=uid={0},ou=people,dc=ibm,dc=com (modify to match with your AD server settings)
   • main.ldapRealm.contextFactory.url=ldap://ldap2.ibm.com:389

   

4. Click on “Save” button from the upper right corner
5. Restart “Knox” service via Ambari Web UI

BigInsights v4.x — Testing Knox Application AD Login with BigInsights Home Page

1. URL: https://<biginsights_home_FQDN>:8443/gateway/default/BigInsightsWeb/index.html
2. Login as a AD user ID and its password