Configure OpenSSH Public Key Authentication with EFS on AIX 6.1.0, TL 4

A step-by-step guide for enabling EFS keystore access while OpenSSH Public Key Authentication is used

OpenSSH is a free tool that includes the implementation of SSH1 and SSH2 protocols. It is a reliable and secure tool that is widely used to replace r-commands. Communication over the ssh session is encrypted and secure as it encrypts all the traffic, including passwords. Prior to this, OpenSSH provided support for opening of the keystore and loading of the keys in the kernel automatically in case of password authentication. But public key authentication does not involve passwords. This article describes how to configure EFS keystore access while OpenSSH Public Key Authentication is used. It explains the procedure for automatic opening of EFS Keystore when ssh public key authentication is used to log on to remote system.

Jyoti B. Tenginakai (, Software engineer, IBM

Photo of Jyoti B. TenginakaiJyoti Tenginakai is a Staff Software Engineer at the IBM India Software labs. She has over two and half years experience with IBM. She currently works with the Security Development team. Before her current assignment, she worked on open source components such as OpenSSH and lsof and was responsible for the release activities, new features, and customer requests and queries for these components. She can be contacted at

Uma Chandolu (, Senior Staff Software Engineer, IBM

Photo of Uma ChandoluUma M. Chandolu works as a Development Support Specialist on AIX. He has five years of extensive hands-on experience in AIX environments and demonstrated expertise in AIX system administration and other subsystems. He has experience interfacing with customers and handling customer-critical situations. He has been recognized as an IBM developerWorks contributing author. He can be contacted at

developerWorks Contributing author

18 May 2010

Also available in Chinese

What is EFS?

In general, the Encrypted Files System (EFS) support on AIX enables individual users on the system to encrypt their data and also access it through keyed protection. Users will be able to setup keys and assign a default key for EFS. These keys are stored in cryptographically protected key store and upon successful login, the user's keys are loaded into the kernel and associated with the kernel processes.

Private keys are associated to users and groups. These keys are stored in keystores and are protected by passwords. A user keystore contains the user's private key and also password to open the user's group keystores; the group keystores contain the groups' private keys.

When a process opens a keystore, either at user login time or using a specific EFS user command, the keys contained in this keystore (and related keystores) are loaded in the kernel and associated with the process credentials. Later on, when the process needs to open an EFS protected file, these credentials are tested. If a key matching the file protection is found, then the process is able to decrypt the file key and therefore the file content.

Keystore creation or opening can happen at login time, by the way of an EFS LAM (old) or PAM (new) module. These modules, as well as the commands (for example, chmod) make calls to some EFS APIs provided by a libefs.a library. Two user commands exist, efsmgr and efskeymgr, to give some control over EFS to the user and administrator.

How to setup Public Key Authentication in OpenSSH

Create a user on the client side and generate keys for this user. Public-private key pairs can be generated using the ssh-keygen command.

  1. On the client side, go to /etc/ssh/ssh_config file and set PubkeyAuthentication yes.
          # hostname
    # grep PubkeyAuthentication /etc/ssh/ssh_config
    PubkeyAuthentication yes

    On the server side, go to /etc/ssh/sshd_config file to set PubkeyAuthentication yes.

            # hostname
     # grep PubkeyAuthentication /etc/ssh/sshd_config
     PubkeyAuthentication yes
  2. Configure OpenSSH server and client to use EFS logon while Public Key Authentication.

    On the client side, go to /etc/ssh/ssh_config file and set "AllowPKCS12keystoreAutoOpen yes".

          # hostname
    # grep AllowPKCS12keystoreAutoOpen /etc/ssh/ssh_config
    AllowPKCS12keystoreAutoOpen yes

    On the server side, go to /etc/ssh/sshd_config file and set "AllowPKCS12keystoreAutoOpen yes".

           # hostname
    # grep AllowPKCS12keystoreAutoOpen /etc/ssh/sshd_config
    AllowPKCS12keystoreAutoOpen yes
  3. Restart the server:
              # hostname
    # stopsrc -s sshd
    0513-044 The sshd Subsystem was requested to stop.
    # startsrc -s sshd
    0513-059 The sshd Subsystem has been started. Subsystem PID is 209040.

    Generate keys with the following command using a user created as follows:

              # hostname
    # mkuser ram
    # su - ram
    # $ ssh-keygen -t rsa -b 2048
    Generating public/private rsa key pair.
    Enter file in which to save the key (/home/ram/.ssh/id_rsa):
    Created directory '/home/ram/.ssh'.
    Enter passphrase (empty for no passphrase):
    Enter same passphrase again:
    Your identification has been saved in /home/ram/.ssh/id_rsa.
    Your public key has been saved in /home/ram/.ssh/
    The key fingerprint is:
    07:5d:0f:20:95:d4:9c:15:8d:77:bd:93:ea:3c:ac:99 ram@ivy02
    The key's randomart image is:
    +--[ RSA 2048]----+
    |        .o+=ooo+.|
    |         o..+o. =|
    |        . .   ..+|
    |         .     + |
    |        S .   . .|
    |         .   .   |
    |            +    |
    |            o=   |
    |           E. .  |

    The command ssh-keygen prompts for passphrase. This passphrase will be used to encrypt the private-key file on the client side. Even ssh-keygen command will accept the empty passphrase, in which case, private-key file will not be encrypted.

    Copy the public keys on to the server in the file ~/.ssh/authorized_keys.

              # hostname
    # cat > /home/laxman/.ssh/authorized_keys
    # cat /home/laxman/.ssh/authorized_keys
    ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAqYK16NpoJ1Nq1/ccb1Ftu2fGkOQd2T4H74dlc6Q

Similarly, any number of a Client user's public key can be copied in the file ~/.ssh/authorized_keys file on server user account.

AIX EFS Configurations

EFS has to be enabled on the server side using the efsenable command. This creates an admin keystore. The keystore gets created for a user in two cases.

  1. Whenever a new user is created.
  2. When passwd is assigned to the user or when user logs in.

The path where user keystore gets created on the system is /var/efs/users/<userlogin>/keystore.

The format of user keystore is in PKCS#12 which contains public and private objects. Private objects are protected by user access key. This user access key is nothing but a hash of a user-defined password (either login password or another password specific to EFS).

Public key cookie needs to be created and inserted into the keystore on server side. User invokes the efskeymgr command to insert the cookie. A public key cookie is the passwd encrypted with users public key.

The following steps show how to create a keystore for a user and insert the public key cookies.

    # hostname

# passwd laxman
laxman's New password:
Enter the new password again:

# ls -l /var/efs/users/laxman
total 8
-rw-------    1 root     system            0 Aug 12 15:40 .lock
-rw-------    1 root     system         1914 Aug 12 15:40 keystore

# su - laxman
$ cd .ssh
$ ls
authorized_keys  id_rsa 

$ efskeymgr -P authorized_keys
laxman's EFS password:

# ls -l /var/efs/users/laxman
total 8
-rw-------    1 root     system            0 Aug 12 15:40 .lock
-rw-------    1 root     system         2252 Aug 12 15:42 keystore

When all the previous configuration setting are complete, run the ssh to log onto the remote machine using the public key authentication.

Run the following command to log on to the remote machine:

# ssh  <username>@<hostname>
  • Once the connection is established and public key authentication is successful, the ssh server checks if AllowPKCS12keystoreAutoOpen is set to 'yes' in the sshd_config file. If so, it sends the ssh client a data packet.
  • The ssh client, on receiving this data packet, checks if the same option is enabled on the client side. That is, the ssh client is configured for this feature by checking if the AllowPKCS12keystoreAutoOpen is set to yes in the ssh_config file. If enabled, the client sends an acknowledgement to the server saying that it too supports this feature.
  • On receiving the ACK from the ssh client, the sshd opens the user's efs keystore in /var/efs/user/<username>/keystore and reads the public key cookie SSHPub(AK) and sends it to ssh client.
  • The ssh client, on receiving the SSHPub from server, decrypts it with its private key and sends the accesskey(AK) back to server. With AK, sshd will open the user's private part of the user's keystore and call the EFS kernel extension to push this opened keystore into the kernel and associate it with the user's log on process.

Verify the authentication and EFS login

The OpenSSH client user ram is all set for Public Key authentication to user laxman on the OpenSSH server with EFS login. Verify the same with ssh login from client:

# hostname

# su - ram

$ ssh -vvv

*                                                                             *
*                                                                             *
*  Welcome to AIX Version 6.1!                                                *
*                                                                             *
*                                                                             *
*  Please see the README file in /usr/lpp/bos for information pertinent to    *
*  this release of the AIX Operating System.                                  *
*                                                                             *
*                                                                             *
$ efskeymgr -V
List of keys loaded in the current process:
 Key #0:
                 Kind ..................... User key
                 Id   (uid / gid) ......... 216
                 Type ..................... Private key
                 Algorithm ................ RSA_1024
                 Validity ................. Key is valid
                 Fingerprint .............. a1a07c79:e0d57e83:8f148a2c:ac778fab:f813cf11


Applications for this setup

This setup can be used along with DB2 UDB DPF for which OpenSSH public key authentication can be used. The DB2 tables are encrypted using EFS.


Check if all the configurations listed above have been performed. Check if the public key cookie is inserted properly by efskeymgr command by verifying the keystore file size before and after the insertion. Enable debug for sshd and check if any failures. Also, verify once with password authentication if the account login and efs login succeed.



Get products and technologies

  • IBM product evaluation versions and get your hands on application development tools and middleware products from DB2®, Lotus®, Rational®, Tivoli®, and WebSphere®.



developerWorks: Sign in

Required fields are indicated with an asterisk (*).

Need an IBM ID?
Forgot your IBM ID?

Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name

The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.


All information submitted is secure.

Dig deeper into AIX and Unix on developerWorks

Zone=AIX and UNIX
ArticleTitle=Configure OpenSSH Public Key Authentication with EFS on AIX 6.1.0, TL 4