Host-based and PAM authentication methods

What's new in IBM-supported OpenSSH


OpenSSH is a free tool that includes the implementation of SSH1 and SSH2 protocols. It is a reliable and secure tool that is widely used to replace r-commands. Communication over the ssh session is encrypted and secure as it encrypts all the traffic, including passwords.

This article describes how to configure host-based and PAM authentication and some of the new features and configuration options added in the IBM-supported version of OpenSSH. See the Related topics section to download the IBM-supported version of OpenSSH.

In order to install OpenSSH, you must first install OpenSSL. It is now available as an installp image format; see the Related topics section for download information.

Host-based authentication in SSH

Host-based authentication in SSH is used when a user on a trusted host wants to log on to a remote machine, which could be an untrusted system. This method does not require a password. Using the following config settings, a user on a trusted host can log onto the remote machine without providing the password.

Enable the following config options on the client and server sides.

  1. On the client machine in the /etc/ssh/ssh_config file set, make the following changes:
    HostbasedAuthentication yes
    EnableSSHKeysign yes
  2. On server side in the /etc/ssh/sshd_config file, make the following changes:
    HostbasedAuthentication yes
    IgnoreRhosts no
  3. Do one of the following:
    • If you are logging on the server machine as root, enter the host in the /.rhosts or /.shosts file.
    • If you are logging on the server machine as a non-root user, enter the host in the /etc/hosts.equiv file.

    The entry pattern must be as follows:

    <client host name or IP Address >   <client username>

    If you want to log in as root on the server machine, run ssh root@< server host name >, and the server checks the /.rhosts or /.shosts file for the client user name and client hostname and allows login only if the correct user name and hostname entries are found.

    If you want to log in as a non-root user on the server machine, run ssh non-root@<server host name>, and the server checks the /etc/hosts.equiv file for the client hostname and client user name entry and allows the user to login.

    When logging in as a non-root user, you should copy the public key on the client machine to the /etc/ssh/ssh_known_hosts or /etc/ssh/ssh_known_hosts2 files. The format of this file should be same as that of the ~/. ssh/known_hosts file. The public key should be preceded by the <client hostname>, <client IP address >.

  4. After modifying the configuration options in the sshd_config file, stop and start the ssh daemon (sshd):
    #stopsrc –s sshd
    0513-044 The sshd Subsystem was requested to stop.
    #startsrc –s sshd
    0513-059 The sshd Subsystem has been started. Subsystem PID is 417812

With these settings, you should be able to log onto the remote machine without prompting for the password.

PAM support in OpenSSH

To enable PAM authentication in OpenSSH, run the following configuration commands:

# lssec -f /etc/security/login.cfg -s usw -a auth_type
usw auth_type=STD_AUTH

If the code looks like this: auth_type = STD_AUTH, then change it to PAM_AUTH using the following command:

# chsec -f /etc/security/login.cfg -s usw -a auth_type=PAM_AUTH

Add the following to the /etc/pam.conf file:

  • In the authentication section:
    sshd    auth    required        /usr/lib/security/pam_aix
  • In account management section:
    sshd    account    required        /usr/lib/security/pam_aix
  • In the password management section:
    sshd    password    required        /usr/lib/security/pam_aix
  • In the session management section:
    sshd    session  required        /usr/lib/security/pam_aix

Once these setting are done, run the ssh command to use the PAM authentication.

What is new in SSH?

The following are some of the new features and config options added in the IBM-supported version of OpenSSH.


Private keys associated with users and groups are stored in the EFS keystore, protected by a keystore password. When a user logs in on an EFS-enabled machine, the login process opens the EFS keystore and the keys contained in this keystore are loaded in the kernel if the user password and keystore password are the same. These keys are used later when a process needs to read or write an EFS-protected file.

With the open source version of SSH (OpenSSH), the login process fails to open the EFS keystore and load the keys in the kernel. This is a limitation of SSH as it spawns two processes, one for authentication and another for session creation. For the EFS keystore to load, creation of the keystore and binding of these keys to process credentials has to be done by a single process. In the case of SSH, two different processes do this, hence loading of the keystore fails.

This problem has been fixed in the IBM-supported version of SSH. There is openssh-4.5p1 support for opening of the keystore and loading the keys in the kernel automatically in case of password authentication, thereby allowing the login process to read or write to the EFS-protected file (provided user login password and EFS keystore password are same).

New sshd_config options

  • ChkHomeDir

    This option has been added in openssh-4.5p1 (IBM-supported version) onwards. It is an option included in the sshd_config file. By default it is "No." When enabled, it is set to "Yes," and it checks for the presence of the home directory of the user. If the user's home directory is not present, it exits and does not allow the user to log in. This option is useful if the systems administrator wants to restrict access to users whose home directory is not present.

  • FrcpasswdPolicies

    This config option is included in the sshd_config file of the IBM-supported version of OpenSSH-4.5p1 onwards. By default, it is set to "No." When it is set to "Yes," it checks whether the password is expired for that user before allowing the user to log in. If the password is expired for the user, it prompts the user to change the password and then allows the user to log in once the password is successfully changed. Otherwise, login fails. This password check is done even if the user is following the public key or host-based authentication. This ensures that the user is authorized before logging in.

  • AllowFiles/DenyFiles

    This option has been added in the sshd_config file of the IBM-supported version of OpenSSH-4.7p1 onwards. This option pertains to SFTP, a secure FTP that allows a user to copy files to and from a remote machine. With these options, the user can control the access of files and directories to copy to or from.

    You can mention the file names in the sshd_config file, as shown in the following example.

    In the sshd_config file, add the following line and save the file:

    AllowFiles  "<jyo>:/tmp/*"

    Run the following command to stop and start the server if the SSH daemon is running:

    # stopsrc -s sshd
    0513-044 The sshd Subsystem was requested to stop.
    # startsrc -s sshd
    0513-059 The sshd Subsystem has been started. Subsystem PID is 11160.

    Run the SFTP command from the client machine as follows:

    # sftp
    Connecting to's password:
    sftp> cd /usr
    Couldn't stat remote file: Permission denied

    In the previous example, after logging in, "jyo" is trying to change to the /usr directory. However, the AllowFiles option for user jyo as "jyo:/tmp/*" in the sshd_config file will not let user "jyo" change any other files other than the those from the /tmp directory.

    Similarly, the DenyFiles option denies access to those files in /directories; the DenyFiles option in sshd_config file allows the remaining files.

    The AllowFiles and DenyFiles options take the following syntax to specify the file and directories names:

    <user/group> : <files/directories>
  • Chroot directory changes

    Follow the instructions in the IBM developerWorks article, openssh with AIX chroot, to learn about the chroot feature inckuded in the IBM-supported version of OpenSSH. See the Related topics section for more a link to this article.

    Starting with OpenSSH-5.0p1, there has been a modification in setting up the chroot environment. A new config option, ChrootDirectory, is included in the sshd_config file that specifies the path to chroot after authentication.

    Setting up the chroot environment remains the same as mentioned in the openssh with AIX chroot developerWorks article, except for the section on "Creating chroot user and finalizing installation." See Related topics. This section can be skipped for OpenSSH-5.0p1 version onwards. Also. make sure that the tmp directory inside the chroot directory has 777 permissions.

    You need to create the home directory of the user in the chroot environment and the chrooted directory should be a root-owned directory.

    For example, take /chroot as the chroot directory, then the sshd_config fille will be as follows:

    ChrootDirectory  /chroot

    The permission for the /chroot directory should be as follows:

    #chown root:system /chroot

    Say you are logging in as a user named "jyoti" in the chrooted environment. The home directory of the user "/home/jyoti" has to be created manually inside /chroot.

    #cd /chroot
    # ls
    bin   dev   etc   home  lib   tmp   unix  usr
    #cd home
    #mkdir  jyoti

    For SFTP to run in the chroot environment, set the Subsystem config option in the sshd_config file as follows:

    Subsystem   sftp     internal-sftp

    This option simplifies the configurations using the ChrootDirectory.

    To restrict the usage of the ChrootDirectory option in the sshd_config file to a particular user, the Match directive can be used for the specific user, as shown:

    Match  User <user name> 
    	ChrootDirectory   <Name of the directory to chroot>
  • Auditing in OpenSSH

    The Auditing subsystem enables the systems administrator to record security-relevant information, which can be analyzed later to detect potential and actual violations of the system's security policy.

    Auditing is supported in IBM-supported versions of SSH from OpenSSH-4.5p1 ( onwards. To enable the auditing in SSH, tweak the following settings.

    Add the following new class to the /etc/security/audit/config file:

    sshcl = SSH_failnone, SSH_failpasswd,SSH_failkbdint,SSH_failpubkey, SSH_failhstbsd,
    SSH_failgssapi, SSH_invldusr,SSH_nologin, SSH_connclose,SSH_auditknwn, SSH_rootdned,
    SSH_exceedmtrix,SSH_connabndn, SSH_authsuccess

    Add the following entry under the "users" section:

    root = sshcl

    Mention the class name (sshcl) for the user you want to enable auditing.

    If the root user already has a set of classes to audit, append sshcl to the end of the list. For example, the root user already has the following:

    root = general,tcpip

    sshcl can be appended to the end of the list:

    root = general,tcpip,sshcl

    In the /etc/security/audit/events file, append the following events:

    SSH_failnone = printf "%s"
    SSH_failpasswd = printf "%s"
    SSH_failkbdint = printf "%s"
    SSH_failpubkey = printf "%s"
    SSH_failhstbsd = printf "%s"
    SSH_failgssapi = printf "%s"
    SSH_invldusr = printf "%s"
    SSH_nologin = printf "%s"
    SSH_connclose = printf "%s"
    SSH_auditknwn = printf "%s"
    SSH_authsuccess = printf "%s"
    SSH_rootdned = printf "%s"
    SSH_exceedmtrix = printf "%s"
    SSH_connabndn = printf "%s"

    Run the following commands to shut down and restart the audit subsystem with the above changes to take effect:

    # /usr/sbin/audit shutdown
    # /usr/sbin/audit  start

    To test to ensure that auditing is configured correctly, run the SSH commands, and then run the auditpr command to get the audit reports:

    auditpr  -v < /audit/trail
  • TCP wrappers support in SSH

    OpenSSH supports TCP wrappers when compiled with the with-tcp-wrappers option. When used, this option restricts the access to TCP services based on the host name or IP address. It uses the /etc/hosts.allow and /etc/hosts.deny configuration files to determine if a client machine is allowed to connect to the SSH server.

    The TCP wrapper service sequentially parses the /etc/hosts.allow file and applies the first rule specified for that service. The pattern to be specified in the file is:

    sshd(Service name) : <host name or IP address>

    The first preference is given to the /etc/hosts.allow file. If a matching rule is found, sshd allows the connection; otherwise, it checks the /etc/hosts.deny file to determine if any matching rule is found. If it finds a matching rule, it denies the connection; otherwise, access is granted.


This article helps users understand how to set up some authentication methods supported by OpenSSH. The article covered configuration to setting up the authentication methods and the new features supported by IBM OpenSSH.

Downloadable resources

Related topic


Sign in or register to add and subscribe to comments.

Zone=AIX and UNIX
ArticleTitle=Host-based and PAM authentication methods