Skip to main content

Is your AIX environment secure?

Find out ways to protect your system from intruders

Shiv Dutta, Technical Consultant, IBM, Software Group
Shiv Dutta works as a Technical Consultant in the IBM Systems and Technology Group where he assists independent software vendors with the enablement of their applications on pSeries servers. Shiv has considerable experience as a software developer, system administrator, and an instructor. He provides AIX support in the areas of system administration, problem determination, performance tuning, and sizing guides. Shiv has worked with AIX from its inception. He holds a Ph.D. in Physics from Ohio University and can be reached at sdutta@us.ibm.com.

Summary:  In this article, the author discusses common security problems in any Unix environment, including AIX. He identifies some of the key files and the steps required to maintain their integrity, and he highlights a number of security tools you can use to diagnose an AIX system and identify potential security lapses. The ideas discussed here apply to all releases of AIX and will be useful to both AIX system administrators and AIX programmers concerned about protecting their systems from intruders.

Date:  20 Dec 2002
Level:  Introductory
Activity:  2643 views
Comments:  

Overview

The underlying philosophy behind the evolution of Unix was convenience. Since security was never a part of the original consideration, no Unix system has ever been truly secure. It was designed by researchers for researchers, and its focus was on easy manipulation of data in a networked, multi-user environment. In the years that followed, certain features have been built into the system to minimize security vulnerabilities, but because of a number of inherent flaws in the operating system itself it is almost impossible to make it fully secure. The following points are worth noting in this context:

  • Despite recent efforts to rectify the situation, the fact is that Unix pretty much works in an all-or-nothing mode. That is, one is either a powerless user, or an all powerful root. Slight lapses in security can in some instances compromise the entire system.
  • A number of functions to manage a Unix system have been implemented outside the kernel and this makes these functions susceptible to easy tampering by anyone who has some degree of Unix expertise.

The existence of security "holes" in Unix is a common knowledge among Unix practioners. Although many of these holes have been plugged, others have popped up from time to time. Sometimes it does seem like we will never be completely immune from hacker's attacks.


Build strong defenses

  • Run tools such as tripwire, crack, and COPS (these tools will be described later in the article) on your systems and monitor the reports generated. A seemingly minor problem in one place, if ignored, may lead to a serious problem elsewhere.
  • Critical files, if kept in the system, must be handled carefully. Since they would attract attention, use of compress followed by crypt could make them relatively secure.
  • Be familiar with the security reports and security patches available from vendors, security mailing lists, and newsgroups and use the knowledge to your advantage.
  • World-writable anonymous ftp directories, shared accounts, and accounts with no passwords or, at best, poorly chosen passwords -- these all open system doors for hackers. Once inside a system, they use it as a base to raid other systems.
  • Check the log files regularly and watch out for odd messages. Also, look out for drastic changes in the activity of an account or activity when there should be none.

What is /etc/passwd file and why is it so important?

This file, known as the password file, is perhaps the most important file as far as Unix security is concerned. It authorizes who can or cannot access the system. Therefore, special care must be taken to ensure that access is not accidentally granted to unintended recipients. Every user account must have a password and the password must be chosen carefully. Entries such as sys, bin, adm, etc. are never needed for login and should be disabled as login accounts.

Later in the article we will discuss a number of security tools that check the password file for potential security problems, but the following one-line simple awk script can be used to find entries that have no passwords.

  awk -F:  '{ if  ($2 == "") print $1 }' /etc/passwd
  

cron could run a script like this on a regular basis and mail the report to the system administrator. It could also be useful to write a script to compare the current version of /etc/passwd against a previous version and have the differences, if any, mailed to the system administrator.

The password file should also be checked for entries that have null or punctuations for user names. Entries such as these will allow logins in many Unix systems including AIX.

No account should be allowed to be shared. In fact, no group logins should be allowed on any system. At many sites, "root" is used as a group login and that is clearly a very bad practice. One could use sudo to control access to services that require root access.

Users should be forced to change their passwords periodically. The root password should be changed more often. The choice of root password should always be amenable to quick typing. It ought to be difficult to guess it simply by watching the finger movements on the keyboard. Watch out for pianists looking over your shoulder. They too have deft fingers!!


Passwords - what to choose or not to choose

The following points should be kept in mind when choosing a password:

  • Should be at least seven characters long.
  • Should include a combination of letters, numbers, punctuation, and changes in case.
  • Should not be a dictionary word.
  • Could be nonsense words, or combination of words.
  • Should not be names or initials.
  • Should not use telephone number or street address.

Where did the password strings disappear?

The password file needs to be world-readable, otherwise commands like ls would not work. This implies that if the password file contained the encrypted password string, it would be available to all users on the system. In principle. it is possible to encrypt a whole dictionary and compare the results with the strings in the password file. If the encrypted strings match, a password has been found.

It is impossible to come up with an eight-character password (only the first eight characters in a password are read by most Unix systems) that can never be cracked, especially these days when the machines are becoming faster and faster. The idea behind a well-chosen password is to make it difficult to crack.

This suggests that user access to encrypted password strings should be restricted. That is the idea behind the /etc/shadow file. The encrypted password strings previously available in the /etc/passwd file are now found in the /etc/shadow file access to which is limited to root. In AIX the corresponding file is /etc/security/passwd.


What gives you the power of root?

The User ID (UID) of the root account, by definition, is zero. Strictly speaking, any account that has the UID of zero is a root account. Since there can be multiple entries in the /etc/passwd file with this UID, there can be multiple ways to log in as root. The hackers use this fact to break into a system. Once they find an authentic root shell, they edit the /etc/passwd file and add accounts with zero UID. Since commands like who and w refer to the account name and not the UID that owns the account, hackers can successfully hide their camouflage even when they are logged in as root.

The /etc/passwd file should therefore be checked on a regular basis for accounts with zero UID. An awk script similar to the one below will do just that and it can be set up to run from cron at regular intervals.

awk -F: ' {if ($3 == 0) print $1}'  /etc/passwd

This script can be modified to find entries with GIDs (Group IDs) or UIDs that are the same as those of key individuals within an organization.


Setuid programs

You have to be especially careful when writing setuid programs. Slight lapses can open doors to critical files and jeopardize system security. Programs written setuid to root are specially vulnerable to major security compromises. It is never safe to take any setuid commands for granted including the ones in Unix distribution.


Handle-with-care files

  • Check the permissions on all files on a regular basis to avoid security lapses. Software included in Unix distributions often have lax file permissions. Unless changed, these permissions may cause a great deal of grief for a production environment.
  • The ps command needs the device file /dev/kmem for it to work. Since this file allows access to the address space of the kernel, a Unix programmer can easily locate encrypted passwords in the kernel's data structures if /dev/kmem was accidentally left as publicly readable.
  • The files /etc/passwd and /etc/group should have mode 644 and should be owned by root. The passwd command runs setuid to root. This is so because otherwise users would need write permission on the password file to change their passwords.
  • One of the common methods used by hackers to distribute files is via publicly writable directories transmitted using anonymous ftp. An ftp site that allows such directories should have a system in place to screen such submissions regularly.
  • Only root should have read and write permissions on device files for hard disk partitions. Read or write permissions on a disk device file essentially amounts to read or write permission on every file in the filesystem it represents. The permissions on these files should therefore be carefully maintained.

/etc/hosts.equiv and ~/.rhosts

These files define which users on foreign hosts are permitted to remotely execute commands on the local host without the need of passwords. The rshd, rlogind, lpd, and srcmstr programs all use these files to determine remote user access.

If you must use these files, make sure that the hosts share login names and UIDs, and only the fully-qualified DNS hostnames are used.


Security and sendmail

Since a large part of Sendmail runs as root, it has been a victim of attacks numerous times. Weaknesses in sendmail security have been well known over the years. It is best to use the latest version of the software. All versions of sendmail released prior to BIND version 8.6.9 (Released April, 1994) have known vulnerabilities.


System diagnosis with security tools

In this section, we will look at a number of tools you can use to monitor the robustness of your system. A detailed discussion of these tools is beyond the scope of this article, but we want to make you aware of their availability and their function. Not all tools need to be installed on the same system. You will have to choose which tools you want to install based on your need. The reports generated by these tools can be utilized to fix the lapses in security and make the system safer from the prying eyes. New tools, if and when they become available, are usually added to the CERT (Computer Emergency Response Team - more about this later) security tools archive in the directory pub/tools at cert.org and announced on the cert-tools-list mailing list.

COPS
Computer Oracle and Password System (COPS) is a set of programs that check files, directories, and device permissions and modes and alerts you to potential problems. Of particular interest to COPS are system startup and crontab files, the system administration files (e.g. /etc/passwd, /etc/group etc.) and the users' home directories. Unfortunately, however, the onus is on the system administrator or the user to fix the highlighted problems.

If installed on a system, COPS will generate a report something like this:

ATTENTION:
Security Report for Sun Nov 1 11:07:00 CST 2002 from host cyclop.austin.ibm.com

Warning!  Password Problem:  Guessed:  tornado  shell:  /bin/ksh
Warning!  Password file, line 8, no password:
Warning!  User sdutta's home directory  /home/sdutta is mode 0777!
Warning!  /var/spool/mail is -World_writable!
Warning! "." (or current directory) is in root's path!
Warning!  Root does not own the following file(s):  /bin

You should run COPS on a nightly basis and have it set up so that the security report is mailed to you. The security "holes,", if any, should be plugged before one of the users gets the idea of running COPS himself and take advantage of the holes.

crack
We have mentioned before the importance of carefully chosen passwords to maintain a secure system environment. The password file is the first line of defense against intruders and, therefore, it's important that the selected passwords be hard to discover. In principle, anybody can discover poorly chosen passwords by comparing the encrypted passwords with an encrypted dictionary. You can prevent this sort of situation by making the comparison yourself before anybody else does and forcing users to change the passwords that have been broken. Crack is a tool that is used for this purpose. It uses a number of common password-guessing techniques to expose poorly-chosen passwords.

tcpd
A tool named tcpd, commonly referred to as the "TCP wrapper," is used by many installations to monitor connections to TCP services such as telnet, rlogin, and finger. It can also be used to control which systems can connect to these services as well as to track or control unwanted guests. tcpd is often used to protect systems from intrusion from the internet.

The implementation of tcpd is trivial. It simply amounts to modifying the /etc/inetd.conf file to execute tcpd instead of the actual program. tcpd takes care of any necessary logging and security checks before the real daemon is executed. For instance, if the /etc/inetd.conf file originally contained the line:

telnet stream  tcp6  nowait  root  /usr/sbin/telnetd  telnetd

Implementation of tcpd would require this line to be changed to:

telnet stream  tcp6  nowait  root  /usr/sbin/tcpd  telnetd

The resulting log file, configured in /etc/syslog.conf, would look something like this:

Nov 01 08:52:43  cyclop telnetd[25880]: connect from homer.austin.ibm.com
Nov 01 09:19:44  cyclop telnetd[15520]: connect from krusty.austin.ibm.com
Nov 01 10:48:45  cyclop telnetd[19332]: connect from marge.austin.ibm.com
Nov 01 11:14:57  cyclop telnetd[2362]: connect from maggie.austin.ibm.com

tripwire
The tool named tripwire can detect system files that have been replaced, corrupted or tampered with. It does this by monitoring the permissions and checksums of these files. For example, tripwire makes it easy to determine that an intruder has replaced the original /etc/passwd with one that has multiple root accounts. tripwire is usually set up to mail a nightly report to the system administrator. A typical tripwire report may look like:

### Phase 1:	Reading configuration file
### Phase 2:	Generating file list
### Phase 3:	Creating file information database
### Phase 4:	Searching for inconsistencies
###
### Total files scanned:	4012
###		Files added:	5 
###		Files deleted:	3
###		Files changed:	56
###
### After applying rules:
###		Changes discarded:	54
###		Changes remaining:	1
###
###
changed:  -rwxr-xr-x root  12483  Nov 01  17:15:27  1998  /etc/passwd
###
### Phase 5:	Generating expect/observed pairs
###
### Attr           Observed (is)                 Expect (should be)
###========        ============                  =================
/etc/passwd
         st_ctime: Nov 05 11:15  2002            Nov 09  13:51  1999
 

In this example, tripwire reports that the inode change time of /etc/passwd is different than on the previous day. This may be an indication that somebody has replaced the vendor's version of /etc/passwd with one of his own.

Kerberos
If all the methods described in the preceding sections were implemented, it is still possible to intrude into a networked machine. Such common practices as transmitting passwords across a network or maintaining a ~.rhosts file in one's home directory can be easy sources of security compromises. The Kerberos system, designed at MIT, attempts to address some of the network security issues in a comprehensive manner.

Kerberos guarantees that users and services are in fact genuine. Kerberos uses the DES encryption algorithm to construct a set of credentials called "tickets". Tickets are passed around the network to certify the user's identity and to provide the user with access to network services. Effectively, Kerberos provides the security of passwords without requiring the user to actually type the password every so often.

In Kerberos, passwords are always transmitted in an encrypted form.

Packet filtering
Packet filters are basic tools to implement security at the network level. A packet filter limits the type of traffic that can pass through a Gateway based on which destination addresses, port numbers, and protocol types are acceptable, and the gateway simply discards any packets that do not meet the profile.

Packet filtering should not be the main line of defense against intruders but only as a supplemental security measure. Every host within the organization should be individually secured with tools such as COPS, crack, tcpd, and tripwire.

crypt
The crypt is a popular utility used to encrypt files. It can also be used to encrypt a whole directory provided it is first packed into a single file by using tar or cpio.

The value of crypt is enhanced if the files are compressed before crypt is used.

crypt requires a "key" that essentially becomes a password for the encrypted file. Unfortunately, however, the key cannot be recovered, so the use of crypt should be taken seriously.

CERT
Computer Emergency Response Team (CERT) was formed by DARPA (Defense Advanced Research Projects Agency). Its Coordination Center is located at CMU's (Carnegie-Mellon University) Software Engineering Institute. Although CERT's charter includes some degree of problem solving, essentially it is a clearing house for vendor security patches and security announcements. These patches and announcements are called "CERT advisories." New advisories are posted to the newsgroup comp.security.announce.

CERT can be reached by email at cert@cert.org.


Security mailing lists and newsgroups

The following mailing lists and Usenet newsgroups are great sources of information to keep abreast of security related problems. Subscription to one or more of these can be quite useful:

NameMailing ListNewsgroupEmail Address or URL
unix-security*security@cpd.com
security-misc*comp.security.misc
virus-list*comp.virus
ACM risks*comp.risks
cert-tools*cert-tools-request@cert.org
cert-advisory*cert-advisory-request@cert.org
security announcements*comp.security.announce

Conclusion

The security tools described here could be set up to be run by cron on a regular basis. This will facilitate identification of security lapses in the system environment and alert the system administrator to take preventive measures. Careful maintenance of the password file and strict enforcement of appropriate file permissions would undoubtedly keep a system fairly secure.


Resources

About the author

Shiv Dutta works as a Technical Consultant in the IBM Systems and Technology Group where he assists independent software vendors with the enablement of their applications on pSeries servers. Shiv has considerable experience as a software developer, system administrator, and an instructor. He provides AIX support in the areas of system administration, problem determination, performance tuning, and sizing guides. Shiv has worked with AIX from its inception. He holds a Ph.D. in Physics from Ohio University and can be reached at sdutta@us.ibm.com.

Comments



Trademarks  |  My developerWorks terms and conditions

Help: Update or add to My dW interests

What's this?

This little timesaver lets you update your My developerWorks profile with just one click! The general subject of this content (AIX and UNIX, Information Management, Lotus, Rational, Tivoli, WebSphere, Java, Linux, Open source, SOA and Web services, Web development, or XML) will be added to the interests section of your profile, if it's not there already. You only need to be logged in to My developerWorks.

And what's the point of adding your interests to your profile? That's how you find other users with the same interests as yours, and see what they're reading and contributing to the community. Your interests also help us recommend relevant developerWorks content to you.

View your My developerWorks profile

Return from help

Help: Remove from My dW interests

What's this?

Removing this interest does not alter your profile, but rather removes this piece of content from a list of all content for which you've indicated interest. In a future enhancement to My developerWorks, you'll be able to see a record of that content.

View your My developerWorks profile

Return from help

static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=AIX and UNIX
ArticleID=13029
ArticleTitle=Is your AIX environment secure?
publish-date=12202002
author1-email=sdutta@us.ibm.com
author1-email-cc=

My developerWorks community

Tags

Help
Use the search field to find all types of content in My developerWorks with that tag.

Use the slider bar to see more or fewer tags.

Popular tags shows the top tags for this particular content zone (for example, Java technology, Linux, WebSphere).

My tags shows your tags for this particular content zone (for example, Java technology, Linux, WebSphere).

Use the search field to find all types of content in My developerWorks with that tag. Popular tags shows the top tags for this particular content zone (for example, Java technology, Linux, WebSphere). My tags shows your tags for this particular content zone (for example, Java technology, Linux, WebSphere).

Special offers