Skip to main content

By clicking Submit, you agree to the developerWorks terms of use.

The first time you sign into developerWorks, a profile is created for you. Select information in your developerWorks profile is displayed to the public, but you may edit the information at any time. Your first name, last name (unless you choose to hide them), and display name will accompany the content that you post.

All information submitted is secure.

  • Close [x]

The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerworks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

By clicking Submit, you agree to the developerWorks terms of use.

All information submitted is secure.

  • Close [x]

Kerberized authentication of Windows Terminal Service

Use IBM Network Authentication Service as your Key Distribution Center on AIX 5.3

Sandeep Ramesh Patil (rsandeep@in.ibm.com), Senior Staff Software Engineer, IBM, Software Group
Photo of Sandeep Patil
Sandeep Ramesh Patil is a Senior Staff Software Engineer for the IBM India Software Labs. He has worked for IBM for the past six years, focusing on distributed technology including DCE, SARPC, and security products such as the IBM Network Authentication Services (IBM Kerberos). He is currently developing new features and implementing security-related RFC for the IBM Network Authentication Service, along with its product support. Sandeep holds a BS degree in computer science and engineering from the University of Pune, India. You can contact him at sandeep.patil@in.ibm.com.
Prashant Sodhiya, Staff Software Engineer, IBM, Software Group
Photo of Prashant Sodhiya
Prashant Sodhiya is a Staff Software Engineer for IBM India Software Labs. He has worked for IBM for the past two years and is currently developing new features and product support for the IBM Network Authentication Service (IBM Kerberos). Before joining IBM, he worked with the Centre for Development of Advanced Computing (CDAC) for three years in network security domain. Prashant holds a BS degree in computer science and engineering from the University of Gwalior, India. You can contact him at psodhiya@in.ibm.com.

Summary:  Discover how to configure the Microsoft® Windows® 2003 Server to authenticate Terminal Service users with the IBM Network Authentication Service (IBM NAS) Key Distribution Center (KDC) being hosted on their AIX® 5.3 system. Such a setup not only gives Kerberized authentication for Terminal Service users, but it also allows users to have uniform user IDs and passwords across AIX and Windows Server systems. It allows application developers to exploit the advantages of Kerberos interoperability between IBM NAS and Windows in Kerberized applications spanning across systems.

Date:  22 Aug 2006
Level:  Intermediate
Also available in:   Chinese  Korean

Activity:  10314 views
Comments:  

Introduction

Kerberos, which provides a secure means of authentication for network users, is one of the most popular authentication mechanisms. Most modern operating systems support Kerberos-based (Version 5) authentication. IBM AIX® 5.3 also supports Kerberos-based authentication.

The IBM version of Kerberos is called IBM Network Authentication Service (IBM NAS), and it can be installed from AIX 5.3 Expansion Pack CDs. IBM NAS for AIX supports both Kerberos clients and Kerberos servers. Many enterprises worldwide use IBM NAS for AIX as the Key Distribution Center (KDC) for their Kerberos realm. It is being used in Network File System (NFS) Version 4 deployment, IBM DB2® Universal Database™ (DB2® UDB™) security, Kerberized AIX integrated login, enterprise-wide authentication, and more.

Today customers generally have a heterogeneous setup, with a mix of UNIX® and Windows® systems. A major challenge for administrators with heterogeneous environments is to have uniform user IDs and passwords across different systems, preferably with a centralized authentication server. Microsoft® Windows Server editions provide a facility called Terminal Services that are widely becoming popular in the Windows world. This facility allows multiple users to log in to a Windows server simultaneously. Microsoft Windows Server editions also support Kerberos-based authentication, which is interoperable with IBM NAS.

In this article, administrators learn how to configure the Microsoft Windows 2003 Server to authenticate Terminal Service users with the IBM NAS KDC being hosted on their AIX 5.3 system. Such a setup not only gives Kerberized authentication for Terminal Service users, but it also allows users to have uniform user IDs and passwords across AIX and Windows Server systems and allows application developers to exploit the advantages of Kerberos interoperability between IBM NAS and Windows in Kerberized applications spanning across systems.

Scenario: IBM NAS KDC on AIX and Kerberized authentication of Windows Terminal Service

We'll use a scenario that takes you through the steps required to set up IBM NAS KDC on an AIX system and have Kerberized authentication of Windows Terminal Service by configuring Windows 2003 Server to IBM NAS KDC.

The following definitions are used in the example in this article:

Kerberos realm name
AIXKERBEROS.IN.IBM.COM
KDC (IBM NAS 1.4)
hostname: fsaix11.in.ibm.com,OS: AIX 5.3
Windows Terminal Service
hostname: windce14.in.ibm.com, OS: Windows 2003 Server (Service Pack 1 with Hotfix for Article ID :902336)
Kerberos administrator name
admin/admin

Figure 1 shows the setup of the example.


Figure 1. Example setup
Example setup

Installing and configuring IBM NAS Server on AIX 5.3

This section covers the installation and configuration of an IBM NAS server (Kerberos KDC) on AIX 5.3.

Installing Kerberos KDC on AIX 5.3

IBM NAS is shipped with AIX 5.3 Expansion Pack CDs. To install the IBM NAS server package, install the krb5.server.rte fileset. You can use the following command to install the NAS server fileset:

[root@fsaix11 / ]# hostname
fsaix11.in.ibm.com

[root@fsaix11 / ]# installp -aqXYgd  .  krb5.server

Then export the following PATH to ensure that you execute IBM NAS commands from the respective IBM NAS directories:

[root@fsaix11 / ]# export PATH=/usr/krb5/sbin:/usr/krb5/bin:$PATH

Configuring Kerberos KDC on AIX 5.3

To configure an IBM NAS server on an AIX machine, use the command in Listing 1 below. In this example, we're using the legacy configuration, where the principals are stored in a database on the local file system. Instead of the legacy configuration, IBM NAS server can also be configured to Lightweight Directory Access Protocol (LDAP) using an LDAP directory plug-in. For more information on configuration of IBM NAS with LDAP, see the IBM NAS Version 1.4 Administration Guide, shipped with AIX Version 5.3 Expansion Pack CD.


Listing 1. Configuring an IBM NAS server on an AIX machine
[root@fsaix11 / ]# hostname
fsaix11.in.ibm.com

[root@fsaix11 / ]# /usr/krb5/sbin/config.krb5 -S -d in.ibm.com -r 
   AIXKERBEROS.IN.IBM.COM 

Initializing configuration...
Creating /etc/krb5/krb5_cfg_type...
Creating /etc/krb5/krb5.conf...
Creating /var/krb5/krb5kdc/kdc.conf...
Creating database files...
Initializing database '/var/krb5/krb5kdc/principal' for realm 'AIXKERBEROS.IN.IBM.COM'
master key name 'K/M@AIXKERBEROS.IN.IBM.COM'
You are prompted for the database Master Password.
It is important that you DO NOT FORGET this password.
Enter database Master Password:
Re-enter database Master Password to verify:
WARNING: no policy specified for admin/admin@AIXKERBEROS.IN.IBM.COM;
  defaulting to no policy. Note that policy may be overridden by
  ACL restrictions.
Enter password for principal "admin/admin@AIXKERBEROS.IN.IBM.COM":
Re-enter password for principal "admin/admin@AIXKERBEROS.IN.IBM.COM":
Principal "admin/admin@AIXKERBEROS.IN.IBM.COM" created.
Creating keytable...
Creating /var/krb5/krb5kdc/kadm5.acl...
Starting krb5kdc...
krb5kdc was started successfully.
Starting kadmind...
kadmind was started successfully.
The command completed successfully.

Because the Windows Kerberos implementation currently supports only DES-CBC-MD5 and DEC-CBC-CRC encryption types, you need to change the IBM NAS Kerberos server default encryption settings so that the Windows workstations can authenticate to an IBM NAS server. You must make the following changes on the AIX machine (fsaix11.in.ibm.com, in your case) hosting the IBM NAS KDC:

  • Edit the /var/krb5/krb5kdc/kdc.conf file and change the value of supported_enctypes to have des-cbc-md5:normal and des-cbc-crc:normal at the beginning of the encryption-type list.
  • After editing, the supported_enctypes section of the /var/krb5/krb5kdc/kdc.conf file should look similar to:
    supported_enctypes = des-cbc-md5:normal des-cbc-crc:normal des3-cbc-sha1:normal 
    arcfour-hmac:normal aes256-cts:normal

Restart the AIX NAS server daemons (for example, krb5kdc and kadmind) so that the above encryption-type changes take effect. To restart the AIX NAS server daemons, use the following commands, as shown in Listing 2.


Listing 2. Restarting the AIX NAS server daemons
[root@fsaix11 / ]# stop.krb5
Stopping /usr/krb5/sbin/krb5kdc...
/usr/krb5/sbin/krb5kdc was stopped successfully.
Stopping /usr/krb5/sbin/kadmind...
/usr/krb5/sbin/kadmind was stopped successfully.
The command completed successfully.
                       
[root@fsaix11 / ]# start.krb5
Starting krb5kdc...
krb5kdc was started successfully.
Starting kadmind...
kadmind was started successfully.
The command completed successfully.

Required Kerberos principals for the Windows Terminal Service users

Now you need to create Kerberos principals corresponding to the Windows Terminal Service users (and services) who wish to have Kerberized authentication over the network. In this setup, you want the "administrator" user of the windce14.in.ibm.com machine hosting the Windows Terminal Service to be authenticated using IBM NAS KDC hosted on fsaix11.in.ibm.com, an AIX 5.3 machine. You are required to create administrator and host/windce14.aixkerberos.in.ibm.com Kerberos principals using the kadmin.local command of IBM NAS on fsaix11.in.ibm.com, as shown below in Listing 3.


Listing 3. The kadmin.local command
[root@fsaix11 / ]# kadmin.local
kadmin.local: ank -pw laurel administrator
WARNING: no policy specified for administrator@AIXKERBEROS.IN.IBM.COM;
  defaulting to no policy. Note that policy may be overridden by
  ACL restrictions.
Principal "administrator@AIXKERBEROS.IN.IBM.COM" created.

kadmin.local:  ank -pw laurel host/windce14.aixkerberos.in.ibm.com
WARNING: no policy specified for 
   host/windce14.aixkerberos.in.ibm.com@AIXKERBEROS.IN.IBM.COM;
  defaulting to no policy. Note that policy may be overridden by
  ACL restrictions.
Principal "host/windce14.aixkerberos.in.ibm.com@AIXKERBEROS.IN.IBM.COM" created.

The administrator is required to create Kerberos principals corresponding to each Windows Terminal Service user that needs Kerberos authentication. In this example, we are demonstrating it only for the "administrator" principal.


Windows 2003 Server Terminal Services readiness

If you have already deployed the Windows 2003 Server Terminal Server in your environment, all you are required to do is install a Microsoft Hotfix for Terminal Services. For Terminal Services to work well with Kerberized authentication configured to IBM NAS KDC on the Windows 2003 Server, you must install a Hotfix provided by Microsoft for the Windows Server 2003-based Terminal Server.

Once you have installed the Hotfix (or the proposed workaround), you are all set to configure the Windows 2003 Server to IBM NAS KDC and run the Windows Terminal Service with Kerberized authentication. For detailed information on installation and configuration of the Microsoft Terminal Server, see the appropriate Microsoft documentation.

Configure Windows 2003 Server (Kerberos client) to IBM NAS server

After installing the Hotfix, you need to configure the Windows 2003 Kerberos client to the IBM NAS server on AIX 5.3. For that, you need to download the Resource Kit Tools from Windows 2003 Server CD, which installs the Windows Kerberos utilities (ksetup, ktpass, and so on).

To configure the Windows 2003 Server to act as a Kerberos client to the IBM NAS server:

  1. Make the Windows Server (windce14.in.ibm.com) a part of your Kerberos workgroup by setting it to your Kerberos domain using the ksetup command:
    C:\>hostname
    windce14
    
    C:\> ksetup /setdomain AIXKERBEROS.IN.IBM.COM
    

  2. Configure the Windows Server machine to the Kerberos realm by specifying the Kerberos realm name and Kerberos server name, as shown below:
    C:\> ksetup /addkdc AIXKERBEROS.IN.IBM.COM fsaix11.in.ibm.com
    

  3. Set the local machine account password, as follows:
    C:\> ksetup /setmachpassword laurel
    

    This password must match the password used when you created the Kerberos host principal (host/windce14.aixkerberos.in.ibm.com) by invoking ank from kadmin.local, explained earlier.

  4. Map the Kerberos user to a local Windows user. The command below maps the local windows administrator user to administrator@AIXKERBEROS.IN.IBM, a Kerberos principal:
      C:\> ksetup /mapuser administrator@AIXKERBEROS.IN.IBM.COM administrator
     

  5. Restart the computer for the changes to take effect.

Figure 2 summarizes all the steps executed above on the Windows machine.


Figure 2. Configuration of Windows 2003 Server as Kerberos client to AIX KDC
Configuration of Windows 2003 Server as Kerberos client to AIX KDC

Testing the setup

You are now all set to exercise the Kerberized authentication of Windows Terminal Service users against IBM NAS KDC hosted on the AIX V 5.3 machine. Log in to the Windows Server machine (windce14.in.ibm.com) using the Remote Desktop Connection from any of your Windows desktop machines.

On connection, it presents you with the logon screen for the windce14.in.ibm.com machine. Select Log on to. You should see that the Kerberos realm you created is also present in the drop-down list. Now enter your Kerberos username and the password (in this case, the username is administrator and the password is laurel), select AIXKERBEROS.IN.IBM.COM(Kerberos Realm) in the "Log on to" option, and select OK. This will then carry out the Kerberized authentication process, and upon success will log the Terminal Service user into the Windows machine.

Figure 3 shows the Remote login to the Windows server machine.


Figure 3. Kerberized authentication of Windows Terminal Service users against IBM NAS KDC
Kerberized authentication of Windows Terminal Service  users against IBM NAS KDC

Conclusion

This article explains how administrators can use the IBM NAS KDC on AIX 5.3 for authentication of Windows 2003 Terminal Service. This should help simplify administration, and it also allows users to have common user IDs and passwords across AIX and Windows Terminal Service systems.


Resources

Learn

Get products and technologies

Discuss

About the authors

Photo of Sandeep Patil

Sandeep Ramesh Patil is a Senior Staff Software Engineer for the IBM India Software Labs. He has worked for IBM for the past six years, focusing on distributed technology including DCE, SARPC, and security products such as the IBM Network Authentication Services (IBM Kerberos). He is currently developing new features and implementing security-related RFC for the IBM Network Authentication Service, along with its product support. Sandeep holds a BS degree in computer science and engineering from the University of Pune, India. You can contact him at sandeep.patil@in.ibm.com.

Photo of Prashant Sodhiya

Prashant Sodhiya is a Staff Software Engineer for IBM India Software Labs. He has worked for IBM for the past two years and is currently developing new features and product support for the IBM Network Authentication Service (IBM Kerberos). Before joining IBM, he worked with the Centre for Development of Advanced Computing (CDAC) for three years in network security domain. Prashant holds a BS degree in computer science and engineering from the University of Gwalior, India. You can contact him at psodhiya@in.ibm.com.

Report abuse help

Report abuse

Thank you. This entry has been flagged for moderator attention.


Report abuse help

Report abuse

Report abuse submission failed. Please try again later.


developerWorks: Sign in


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Select information in your developerWorks profile is displayed to the public, but you may edit the information at any time. Your first name, last name (unless you choose to hide them), and display name will accompany the content that you post.

Choose your display name

The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


Rate this article

Comments

Help: Update or add to My dW interests

What's this?

This little timesaver lets you update your My developerWorks profile with just one click! The general subject of this content (AIX and UNIX, Information Management, Lotus, Rational, Tivoli, WebSphere, Java, Linux, Open source, SOA and Web services, Web development, or XML) will be added to the interests section of your profile, if it's not there already. You only need to be logged in to My developerWorks.

And what's the point of adding your interests to your profile? That's how you find other users with the same interests as yours, and see what they're reading and contributing to the community. Your interests also help us recommend relevant developerWorks content to you.

View your My developerWorks profile

Return from help

Help: Remove from My dW interests

What's this?

Removing this interest does not alter your profile, but rather removes this piece of content from a list of all content for which you've indicated interest. In a future enhancement to My developerWorks, you'll be able to see a record of that content.

View your My developerWorks profile

Return from help

static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=AIX and UNIX
ArticleID=155482
ArticleTitle=Kerberized authentication of Windows Terminal Service
publish-date=08222006
author1-email=rsandeep@in.ibm.com
author1-email-cc=
author2-email=psodhiya@in.ibm.com
author2-email-cc=

Tags

Help
Use the search field to find all types of content in My developerWorks with that tag.

Use the slider bar to see more or fewer tags.

For articles in technology zones (such as Java technology, Linux, Open source, XML), Popular tags shows the top tags for all technology zones. For articles in product zones (such as Info Mgmt, Rational, WebSphere), Popular tags shows the top tags for just that product zone.

For articles in technology zones (such as Java technology, Linux, Open source, XML), My tags shows your tags for all technology zones. For articles in product zones (such as Info Mgmt, Rational, WebSphere), My tags shows your tags for just that product zone.

Use the search field to find all types of content in My developerWorks with that tag. Popular tags shows the top tags for this particular content zone (for example, Java technology, Linux, WebSphere). My tags shows your tags for this particular content zone (for example, Java technology, Linux, WebSphere).

Try IBM PureSystems. No charge.

Special offers