Security is hard. It doesn't sit still, and it's difficult to know how far it needs to extend: if you don't watch yourself, you can end up believing that your boss needs to understand the beauty of elliptic curves when all he really wants is to make sure the custodian doesn't read his annual budget.
However challenging it is to keep up with all facets of computing security, a few areas have matured enough to be worth learning methodically. The first one I recommend to anyone working with Linux servers is account management.
Many of the first wave of books devoted to Linux administration and programming included a chapter on "user management" or "account management." Their meaning was rather specific: how to set up and maintain the computing accounts and group affiliations for the people who use your host.
At that time, "use" necessarily meant "log in to." Account management was all about working with commands such as
chsh, and so on, to configure Linux accounts that would be convenient for a user population dominated by fellow developers. /etc/passwd and its APIs were the focus of Linux experts.
Those times are long past, and the first recommendation I make for most servers is to eliminate most of /etc/passwd. What I mean is this: For historical reasons, most e-mail servers, Web servers, file servers, and so on, manage their user access in terms of /etc/passwd. I think this is generally a mistake. It was a sensible practice earlier in our history, when one or two dozen engineers might share a high-end workstation. Conventional /etc/passwd practices are a mistake, though, when one e-mail server might handle mailboxes for tens of thousands of users, for most of whom computing is just another utility like the water fountain or telephone system.
It's certainly possible to rely on /etc/passwd. It's been patched and tweaked enough to handle surprising workloads. It shouldn't have to, though. If you move user accounts into a dedicated datastore, such as an LDAP (lightweight directory access protocol) or even an RDBMS (relational database management system) datastore, you gain advantages in scalability, security, and maintenance. Restrict /etc/passwd to the few developers and administrators who truly need logins.
This practice gives a big advantage in security, because the duty cycles of service (e-mail, Web, and so on) users are entirely different from those of developers. Once you have burned in a new server, its /etc/passwd shouldn't change often. It's an easy job to monitor it for any updates and, particularly, for tampering. If you're running a large server, though, you might have several new and expired e-mail account changes daily. You need to isolate those from the wider access that /etc/passwd gives.
Is construction of an alternate account datastore a serious recommendation? Yes it is, as surprising as that may seem. A lot of effort has gone in over the years to make very large /etc/passwds, filled mostly with login-free users, work properly. If you do decide to code your own account authentication, and you rely on such traditional e-mail programs as
sendmail, you might well find yourself writing changes for SMTP, POP3, and IMAP4 servers.
Those obstacles generally incline developers toward using off-the-shelf software. My habit is to favor solutions others have written and that I can re-use. One difference with these industrial-use servers, though, is that I often need to customize them anyway -- to set up special message directories, logging information, or usage accounting, for example. What decides the issue for me is a preference to modularize security considerations. I like to be able to manage developer and administrator accounts entirely separately from end-user services. By splitting off the latter from /etc/passwd, I can easily lock down either side without affecting the other.
Almost as important as separating developer accounts from user services is to automate policy. Establish specific, detailed processes for creating and deleting accounts, both developer (/etc/passwd) and end-user (e-mail, Web, database, and so on). While it's good discipline to capture these into executables, it's not strictly necessary. What is important is that the processes be understandable and unambiguous. Casual account creation and deletion always leaves security holes. Review your processes with your human resources, customer support, or other pertinent departments. It's hard to appreciate how crucial this is unless you've lived through the alternative.
When you don't have written procedures for adding and removing users, the invariable result is that a new worker shows up on Monday, say, and by Friday still can't get to his or her company files. Or someone resigns, says goodbye at the holiday party, and is still retrieving occasional corporate assets as February begins.
One of the incidental benefits of account automation is that it encourages more thorough validation. If developers don't have a convenient way to configure accounts with different properties, they're quite unlikely to exercise applications that are supposed to differentiate those configurations.
I recently experienced this first-hand. I was called in on an emergency in which our implementation team had "correctly," in effect, allowed managers to review employee performance reviews -- even for employees they didn't manage! As ridiculous as that sounds, it's typical of security issues. It had even been flagged a couple of times during analysis and design reviews. Every time it was brought to a decision-maker, though, it was part of a sufficiently large and muddy collection of problems that it was passed on without crisp resolution.
Only when a support specialist finally set up a concrete example of a general instance -- one with multiple managers, each of whom had multiple employees reporting in -- did the error get the attention it deserved. Save yourself last-minute dramas; make configuration of all sorts of user accounts routine and available for thorough testing.
The hardest part of security, at least for many of us, is to avoid doing dumb things. Security is one of the "weakest link" affairs, where a single loophole can make all the rest of your investment, however huge and well-planned, laughably pointless. To do security well, you have to stay on the alert for things that you aren't otherwise thinking about.
U.S. government sites frequently exemplify the magnitude of that challenge. One federal agency, frequently in the news for security issues in the "anti-terrorist" sense, maintains a Web site where user passwords are displayed in the open, on the page for changing user preferences. Quite a few organizations address the frequency of lost passwords by assigning passwords based on more-or-less public information (for example, "your password is the first four letters of your birthplace, followed by the final two digits of your birthyear").
How can you avoid such catastrophic mistakes? Unfortunately, there are few systematic ways to succeed at such an abstract goal as "being smart." Among the useful steps to take, though, are study of the RISKS digest and disciplined engineering review.
RISKS is an online newsletter that Peter G. Neumann has been editing since 1985 (see Resources below). Reading it is excellent practice in thinking about how things -- security on your Linux servers, in particular -- can go wrong. Neumann makes the digest quite readable and entertaining, if occasionally macabre.
You should also acquire the habit of trying out your ideas on others. You might think of "software inspection" as no more than a way to spot misplaced punctuation in developers' source code, but it's actually a far more interesting and productive practice. In particular, inspections are a great way to organize peer reviews of requirements documents, Web sites, and all sorts of other artifacts. Stage inspections. See your work through the eyes of others. You'll probably learn a lot about the security, or insecurity, of your servers.
The Resources point to more reading on the subjects of hardening servers, Linux security, and inspections. While server security is an enormous subject, you can pursue the aspects this column describes quickly and inexpensively. If you haven't looked into these approaches already, you should; they'll improve the security of your operations a great deal.
What problems of server security are hardest for you? "Server clinic" will return to the security topic at least a couple more times in the coming year. If you write to me or post your thoughts in our forum, I'll try to address your issues.
- Participate in the discussion forum.
- Check out the other installments of Server clinic.
- "General System Security" is Chapter 5 from the online Hands-on Guide to Red Hat Linux.
- "OpenSSH key management, Part 1" is a good introduction to practical
sshuse (developerWorks, July 2001).
- "Addressing security issues in Linux" is an IBM whitepaper that discusses various security issues, outlines strategies for dealing with them, and lists IBM as well as non-IBM products that can help keep systems buttoned up.
- The RISKS Digest page provides access to subscription information and indexes to back issues of The RISKS Digest.
LinuxSecurity.com is a community resource that points to valuable information.
- The book by Limoncelli and Hogan, which was the subject of an earlier "Server clinic" column, provides an excellent guide to thinking about security beyond the details of one particular operating system.
is a marvelous book on the subject. The discipline applies far more broadly than to just the "code reviews" the title brings to many programmers' minds.
- The Software Engineering Institute formally defines software inspections at its site.
- The SANS Institute is the ideal place to locate resources having to do with system administration, networking, and security.
- The IBM Security Web site contains an overview of security solutions, hardware/software, products, research technology, announcements, press releases, and more.
- Find more Linux articles in the developerWorks Linux zone.