Kerberized authentication of Windows Terminal Service

Use IBM Network Authentication Service as your Key Distribution Center on AIX 5.3

Discover how to configure the Microsoft® Windows® 2003 Server to authenticate Terminal Service users with the IBM Network Authentication Service (IBM NAS) Key Distribution Center (KDC) being hosted on their AIX® 5.3 system. Such a setup not only gives Kerberized authentication for Terminal Service users, but it also allows users to have uniform user IDs and passwords across AIX and Windows Server systems. It allows application developers to exploit the advantages of Kerberos interoperability between IBM NAS and Windows in Kerberized applications spanning across systems.

Sandeep Ramesh Patil (rsandeep@in.ibm.com), Senior Staff Software Engineer, IBM

Photo of Sandeep PatilSandeep Ramesh Patil is a Senior Staff Software Engineer for the IBM India Software Labs. He has worked for IBM for the past six years, focusing on distributed technology including DCE, SARPC, and security products such as the IBM Network Authentication Services (IBM Kerberos). He is currently developing new features and implementing security-related RFC for the IBM Network Authentication Service, along with its product support. Sandeep holds a BS degree in computer science and engineering from the University of Pune, India. You can contact him at sandeep.patil@in.ibm.com.


developerWorks Master author
        level

Prashant Sodhiya, Staff Software Engineer, IBM

Photo of Prashant SodhiyaPrashant Sodhiya is a Staff Software Engineer for IBM India Software Labs. He has worked for IBM for the past two years and is currently developing new features and product support for the IBM Network Authentication Service (IBM Kerberos). Before joining IBM, he worked with the Centre for Development of Advanced Computing (CDAC) for three years in network security domain. Prashant holds a BS degree in computer science and engineering from the University of Gwalior, India. You can contact him at psodhiya@in.ibm.com.



22 August 2006

Also available in Chinese

Introduction

Kerberos, which provides a secure means of authentication for network users, is one of the most popular authentication mechanisms. Most modern operating systems support Kerberos-based (Version 5) authentication. IBM AIX® 5.3 also supports Kerberos-based authentication.

The IBM version of Kerberos is called IBM Network Authentication Service (IBM NAS), and it can be installed from AIX 5.3 Expansion Pack CDs. IBM NAS for AIX supports both Kerberos clients and Kerberos servers. Many enterprises worldwide use IBM NAS for AIX as the Key Distribution Center (KDC) for their Kerberos realm. It is being used in Network File System (NFS) Version 4 deployment, IBM DB2® Universal Database™ (DB2® UDB™) security, Kerberized AIX integrated login, enterprise-wide authentication, and more.

Today customers generally have a heterogeneous setup, with a mix of UNIX® and Windows® systems. A major challenge for administrators with heterogeneous environments is to have uniform user IDs and passwords across different systems, preferably with a centralized authentication server. Microsoft® Windows Server editions provide a facility called Terminal Services that are widely becoming popular in the Windows world. This facility allows multiple users to log in to a Windows server simultaneously. Microsoft Windows Server editions also support Kerberos-based authentication, which is interoperable with IBM NAS.

In this article, administrators learn how to configure the Microsoft Windows 2003 Server to authenticate Terminal Service users with the IBM NAS KDC being hosted on their AIX 5.3 system. Such a setup not only gives Kerberized authentication for Terminal Service users, but it also allows users to have uniform user IDs and passwords across AIX and Windows Server systems and allows application developers to exploit the advantages of Kerberos interoperability between IBM NAS and Windows in Kerberized applications spanning across systems.

Scenario: IBM NAS KDC on AIX and Kerberized authentication of Windows Terminal Service

We'll use a scenario that takes you through the steps required to set up IBM NAS KDC on an AIX system and have Kerberized authentication of Windows Terminal Service by configuring Windows 2003 Server to IBM NAS KDC.

The following definitions are used in the example in this article:

Kerberos realm name
AIXKERBEROS.IN.IBM.COM
KDC (IBM NAS 1.4)
hostname: fsaix11.in.ibm.com,OS: AIX 5.3
Windows Terminal Service
hostname: windce14.in.ibm.com, OS: Windows 2003 Server (Service Pack 1 with Hotfix for Article ID :902336)
Kerberos administrator name
admin/admin

Figure 1 shows the setup of the example.

Figure 1. Example setup
Example setup

Installing and configuring IBM NAS Server on AIX 5.3

This section covers the installation and configuration of an IBM NAS server (Kerberos KDC) on AIX 5.3.

Installing Kerberos KDC on AIX 5.3

IBM NAS is shipped with AIX 5.3 Expansion Pack CDs. To install the IBM NAS server package, install the krb5.server.rte fileset. You can use the following command to install the NAS server fileset:

[root@fsaix11 / ]# hostname
fsaix11.in.ibm.com

[root@fsaix11 / ]# installp -aqXYgd  .  krb5.server

Then export the following PATH to ensure that you execute IBM NAS commands from the respective IBM NAS directories:

[root@fsaix11 / ]# export PATH=/usr/krb5/sbin:/usr/krb5/bin:$PATH

Configuring Kerberos KDC on AIX 5.3

To configure an IBM NAS server on an AIX machine, use the command in Listing 1 below. In this example, we're using the legacy configuration, where the principals are stored in a database on the local file system. Instead of the legacy configuration, IBM NAS server can also be configured to Lightweight Directory Access Protocol (LDAP) using an LDAP directory plug-in. For more information on configuration of IBM NAS with LDAP, see the IBM NAS Version 1.4 Administration Guide, shipped with AIX Version 5.3 Expansion Pack CD.

Listing 1. Configuring an IBM NAS server on an AIX machine
[root@fsaix11 / ]# hostname
fsaix11.in.ibm.com

[root@fsaix11 / ]# /usr/krb5/sbin/config.krb5 -S -d in.ibm.com -r 
   AIXKERBEROS.IN.IBM.COM 

Initializing configuration...
Creating /etc/krb5/krb5_cfg_type...
Creating /etc/krb5/krb5.conf...
Creating /var/krb5/krb5kdc/kdc.conf...
Creating database files...
Initializing database '/var/krb5/krb5kdc/principal' for realm 'AIXKERBEROS.IN.IBM.COM'
master key name 'K/M@AIXKERBEROS.IN.IBM.COM'
You are prompted for the database Master Password.
It is important that you DO NOT FORGET this password.
Enter database Master Password:
Re-enter database Master Password to verify:
WARNING: no policy specified for admin/admin@AIXKERBEROS.IN.IBM.COM;
  defaulting to no policy. Note that policy may be overridden by
  ACL restrictions.
Enter password for principal "admin/admin@AIXKERBEROS.IN.IBM.COM":
Re-enter password for principal "admin/admin@AIXKERBEROS.IN.IBM.COM":
Principal "admin/admin@AIXKERBEROS.IN.IBM.COM" created.
Creating keytable...
Creating /var/krb5/krb5kdc/kadm5.acl...
Starting krb5kdc...
krb5kdc was started successfully.
Starting kadmind...
kadmind was started successfully.
The command completed successfully.

Because the Windows Kerberos implementation currently supports only DES-CBC-MD5 and DEC-CBC-CRC encryption types, you need to change the IBM NAS Kerberos server default encryption settings so that the Windows workstations can authenticate to an IBM NAS server. You must make the following changes on the AIX machine (fsaix11.in.ibm.com, in your case) hosting the IBM NAS KDC:

  • Edit the /var/krb5/krb5kdc/kdc.conf file and change the value of supported_enctypes to have des-cbc-md5:normal and des-cbc-crc:normal at the beginning of the encryption-type list.
  • After editing, the supported_enctypes section of the /var/krb5/krb5kdc/kdc.conf file should look similar to:
    supported_enctypes = des-cbc-md5:normal des-cbc-crc:normal des3-cbc-sha1:normal 
    arcfour-hmac:normal aes256-cts:normal

Restart the AIX NAS server daemons (for example, krb5kdc and kadmind) so that the above encryption-type changes take effect. To restart the AIX NAS server daemons, use the following commands, as shown in Listing 2.

Listing 2. Restarting the AIX NAS server daemons
[root@fsaix11 / ]# stop.krb5
Stopping /usr/krb5/sbin/krb5kdc...
/usr/krb5/sbin/krb5kdc was stopped successfully.
Stopping /usr/krb5/sbin/kadmind...
/usr/krb5/sbin/kadmind was stopped successfully.
The command completed successfully.
                       
[root@fsaix11 / ]# start.krb5
Starting krb5kdc...
krb5kdc was started successfully.
Starting kadmind...
kadmind was started successfully.
The command completed successfully.

Required Kerberos principals for the Windows Terminal Service users

Now you need to create Kerberos principals corresponding to the Windows Terminal Service users (and services) who wish to have Kerberized authentication over the network. In this setup, you want the "administrator" user of the windce14.in.ibm.com machine hosting the Windows Terminal Service to be authenticated using IBM NAS KDC hosted on fsaix11.in.ibm.com, an AIX 5.3 machine. You are required to create administrator and host/windce14.aixkerberos.in.ibm.com Kerberos principals using the kadmin.local command of IBM NAS on fsaix11.in.ibm.com, as shown below in Listing 3.

Listing 3. The kadmin.local command
[root@fsaix11 / ]# kadmin.local
kadmin.local: ank -pw laurel administrator
WARNING: no policy specified for administrator@AIXKERBEROS.IN.IBM.COM;
  defaulting to no policy. Note that policy may be overridden by
  ACL restrictions.
Principal "administrator@AIXKERBEROS.IN.IBM.COM" created.

kadmin.local:  ank -pw laurel host/windce14.aixkerberos.in.ibm.com
WARNING: no policy specified for 
   host/windce14.aixkerberos.in.ibm.com@AIXKERBEROS.IN.IBM.COM;
  defaulting to no policy. Note that policy may be overridden by
  ACL restrictions.
Principal "host/windce14.aixkerberos.in.ibm.com@AIXKERBEROS.IN.IBM.COM" created.

The administrator is required to create Kerberos principals corresponding to each Windows Terminal Service user that needs Kerberos authentication. In this example, we are demonstrating it only for the "administrator" principal.


Windows 2003 Server Terminal Services readiness

If you have already deployed the Windows 2003 Server Terminal Server in your environment, all you are required to do is install a Microsoft Hotfix for Terminal Services. For Terminal Services to work well with Kerberized authentication configured to IBM NAS KDC on the Windows 2003 Server, you must install a Hotfix provided by Microsoft for the Windows Server 2003-based Terminal Server.

Once you have installed the Hotfix (or the proposed workaround), you are all set to configure the Windows 2003 Server to IBM NAS KDC and run the Windows Terminal Service with Kerberized authentication. For detailed information on installation and configuration of the Microsoft Terminal Server, see the appropriate Microsoft documentation.

Configure Windows 2003 Server (Kerberos client) to IBM NAS server

After installing the Hotfix, you need to configure the Windows 2003 Kerberos client to the IBM NAS server on AIX 5.3. For that, you need to download the Resource Kit Tools from Windows 2003 Server CD, which installs the Windows Kerberos utilities (ksetup, ktpass, and so on).

To configure the Windows 2003 Server to act as a Kerberos client to the IBM NAS server:

  1. Make the Windows Server (windce14.in.ibm.com) a part of your Kerberos workgroup by setting it to your Kerberos domain using the ksetup command:
    C:\>hostname
    windce14
    
    C:\> ksetup /setdomain AIXKERBEROS.IN.IBM.COM
  2. Configure the Windows Server machine to the Kerberos realm by specifying the Kerberos realm name and Kerberos server name, as shown below:
    C:\> ksetup /addkdc AIXKERBEROS.IN.IBM.COM fsaix11.in.ibm.com
  3. Set the local machine account password, as follows:
    C:\> ksetup /setmachpassword laurel

    This password must match the password used when you created the Kerberos host principal (host/windce14.aixkerberos.in.ibm.com) by invoking ank from kadmin.local, explained earlier.

  4. Map the Kerberos user to a local Windows user. The command below maps the local windows administrator user to administrator@AIXKERBEROS.IN.IBM, a Kerberos principal:
      C:\> ksetup /mapuser administrator@AIXKERBEROS.IN.IBM.COM administrator
  5. Restart the computer for the changes to take effect.

Figure 2 summarizes all the steps executed above on the Windows machine.

Figure 2. Configuration of Windows 2003 Server as Kerberos client to AIX KDC
Configuration of Windows 2003 Server as Kerberos client to AIX KDC

Testing the setup

You are now all set to exercise the Kerberized authentication of Windows Terminal Service users against IBM NAS KDC hosted on the AIX V 5.3 machine. Log in to the Windows Server machine (windce14.in.ibm.com) using the Remote Desktop Connection from any of your Windows desktop machines.

On connection, it presents you with the logon screen for the windce14.in.ibm.com machine. Select Log on to. You should see that the Kerberos realm you created is also present in the drop-down list. Now enter your Kerberos username and the password (in this case, the username is administrator and the password is laurel), select AIXKERBEROS.IN.IBM.COM(Kerberos Realm) in the "Log on to" option, and select OK. This will then carry out the Kerberized authentication process, and upon success will log the Terminal Service user into the Windows machine.

Figure 3 shows the Remote login to the Windows server machine.

Figure 3. Kerberized authentication of Windows Terminal Service users against IBM NAS KDC
Kerberized authentication of Windows Terminal Service users against IBM NAS KDC

Conclusion

This article explains how administrators can use the IBM NAS KDC on AIX 5.3 for authentication of Windows 2003 Terminal Service. This should help simplify administration, and it also allows users to have common user IDs and passwords across AIX and Windows Terminal Service systems.

Resources

Learn

Get products and technologies

Discuss

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into AIX and Unix on developerWorks


static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=AIX and UNIX
ArticleID=155482
ArticleTitle=Kerberized authentication of Windows Terminal Service
publish-date=08222006