Contents: Adopting a proactive approach to security Why you should design for security How to assess the security risks Developing and ranking security requirements The importance of external analysis Security testing versus functional testing Dynamic testing versus static testing Conclusion Resources About the authors Rate this article Subscriptions: dW newsletters dW Subscription (CDs and downloads)

Gary McGraw, Reliable Software Technologies
John Viega, Reliable Software Technologies

01 Feb 2000

The Internet has introduced a major change in the way companies do business. Electronic commerce over the Internet is showing tremendous growth, with no sign of slowing down. As companies both large and small invest increasingly greater resources into the e-commerce side of their business, security should become a correspondingly bigger concern. When bits mean money, protecting those bits through secure software suddenly becomes an important aspect of running a successful business.

One of the biggest problems in computer security these days is that software is not robust enough in general. Software failures can have disastrous real-world consequences, including a big red mark on the fiscal health of a business.

Security is only one set of software failures that can cause significant financial harm -- all the more so in the accelerated world of electronic commerce on the Internet. Other software failures include reliability and safety problems. Software has in some cases cost people their livelihoods, and even their lives; it is only a matter of time before inadequately secure software causes the decline or death of a large company.

Everyone seems to know or suspect that these problems exist, but few have the know-how to overcome them. Not many resources are available to teach developers how to write secure code. To make matters worse, much of today's software is developed at incredible speed and under intense market pressure. Often the first thing to suffer under such market pressure is software quality (of any sort). Security tends to be an afterthought at best, and is often forgotten completely.

That is not only a tragedy, but a recipe for disaster.

Adopting a proactive approach to security
Retrofitting security is always the wrong solution. Instead, security must be designed into software from the beginning. At Reliable Software Technologies, we have developed a software assurance methodology for security that we have been using successfully for several years in security consulting, code development, and security analysis of code. This methodology has proven its usefulness time and time again in real-world experience -- though it is not guaranteed to solve all your problems with minimal effort.

Security is a complex field, and developing secure applications is no exception. A chief goal of our methodology is to help developers avoid the ad hoc "penetrate and patch" approach to security, where bugs are repaired when the developer learns about them, but otherwise security is not a consideration. (We discussed the many disadvantages of this approach in our previous column, "Making software behave".)

Our approach is a five-step process:

1. Design a system with security in mind.
2. Analyze the system in light of known and anticipated risks.
3. Rank the risks according to their severity.
4. Test for the risks.
5. Cycle a broken system back through the design process.

One subtle yet important difference exists between our approach and the penetrate-and-patch techniques: Our methodology encourages you to test and fix your software for security before the crackers get a chance to do so. Even if we do that, however, we’re still somewhat practicing penetrate-and-patch. The real difference is that we try to do all the penetrating and patching before the software is ever released. In the attempt to eliminate bugs in advance, our methodology demands that a developer be well educated about all potential security risks. In addition, it also leans heavily on good software engineering techniques.

This is all well and good, but one important commonality exists as well. The common thread between all five features of our methodology is that none of them come easy. To get the most out of this methodology, it is critical that you keep yourself informed on all vital industry issues.

Why you should design for security
In contrast to the ad hoc security development techniques, which do not tend to work very well, security should be considered during all phases of the development cycle, instead of bolted on as an afterthought. Retrofitting security onto an existing system is simply a bad idea. Security is not a basic feature that you can add to a system at any time. Security is like fault tolerance; it is a system-wide emergent property that requires significant and careful planning and design.

A better approach to writing secure software is to design security into a system from the very beginning. We know this from many examples of systems that were not originally designed with security in mind, but where security features were added later. Many of these systems are security nightmares.

One such example is the Windows 95 and Windows 98 platform, which is notorious for its numerous security vulnerabilities. It is generally believed that any Windows 9x machine on a network can be crashed or hacked by a knowledgeable attacker. For example, authentication mechanisms in Windows 9x are quite prone to being defeated. The biggest problem is that anyone can sit down at a Windows 9x machine, shut the machine off, turn it back on, log in without a password, and have full control over the computer. In short, the Windows operating system was not designed for today's networking environment; it was designed when PCs were standalone machines. Microsoft's attempts to retrofit its operating system to provide security for this new type of computer use have not been very successful.

UNIX, which was developed by and for university researchers, was not designed with security in mind either. It was meant as a platform for sharing research results among groups of team players. Because of this, it also suffers from enormous amounts of patching and security retrofitting; as with Microsoft, these efforts have not been very successful.

We have seen many real-world systems (designed for use on protected proprietary networks) that are similarly being reworked for use over the Internet. In every case, Internet-specific risks cause the systems to lose all their security properties.

Some people refer to this as an environment problem, where a system that is secure enough in one environment is completely insecure when placed in another. As the world becomes more interconnected via the Internet, however, the environment most machines must find themselves in is, at times, less than friendly.

It is always better to design for security from scratch than to try to add security to an existing design. Reuse is an admirable goal, but the environment in which a system will be used is so integral to security that any change of environment is likely to cause all sorts of trouble -- so much trouble that well tested and well understood things fall to pieces.

Security should be a high priority for software developers because it is an issue of trust and, ultimately, business stability for you and your customers.

How to assess the security risks
There is a fundamental tradeoff between an application's functionality and its security. Generally, you simply cannot be sure that your system is completely secure. A common joke goes that the most secure computer in the world is one that is buried in a ten-foot hole filled with concrete. Good luck in getting much use out of such a machine! With real-world systems, the security question boils down to how much risk a given enterprise is willing to assume in order to solve the problem at hand effectively. Security is really a question of risk management.

The first step in risk management is assessing risk: identifying the potential risks, their likelihood, and their potential severity.

Effective risk assessment requires an expert knowledge of security. The assessor must be able to identify situations where known attacks could potentially be applied to the system at hand, since few totally unique attacks ever rear their heads. Unfortunately, such expert knowledge is hard to come by. Software security is a big problem that currently no single source explains in one place. (In time, we hope this column will help solve that problem.)

Risk identification works best when you have a detailed specification of a system from which to work. It is invaluable to have a definitive resource to answer questions about how the system will be expected to act under particular circumstances. When the specification is in the developer's head, and not on paper, the whole process becomes much more fuzzy. It is easy to consult your mental requirements twice and get contradictory information without realizing it.

Once risks have been identified, the next step is to rank them in order of severity. The relative severity of risks depends greatly on the needs and goals of the system at hand. Thus, in this step, it helps to consult a detailed requirements document. Some risks may not be worth mitigating at all, because the level of risk is small, or the ill effects of a successful attack are no great concern.

Risk assessment is critical for determining how to allocate testing and analysis resources. Since resource allocation is a business problem, having sound data helps ensure good business decisions pertaining to resource allocation.

Developing and ranking security requirements
We have already hinted that good software engineering practices are important to any sound security methodology. Any system that is designed according to well understood and well documented requirements will be superior to a system that is cobbled together out of chewing gum and bailing wire.

You should be sure to craft your requirements well. For example, the requirement "This application should use cryptography wherever necessary" is a poor one, as it prescribes a solution without even diagnosing the problem. The requirements document should communicate not only what the system must do and must not do, but also why the system should behave as described. A much better requirement in this example would be " Credit card numbers being sensitive information, they should be protected against potential eavesdropping." The choice of how to protect the information -- cryptography or some other means -- should be deferred until the system has a specification.

We recommend creating a template or document of standard security guidelines for your organization from which you derive security-related requirements for any particular project. Such a document allows for individual applications with individual needs, as well as different priorities on different security concerns, while providing a framework that allows for a consistent analysis across applications. For example, denial of service may not be a significant concern for a client application, as only the client is affected. But a denial of service attack on a commercial Web server could potentially deny service to thousands of people.

Such guidelines generally consist of both a general explanation of how to perform a security analysis, and a list of risks that application developers should be sure to consider. Of course, the developer should not expect such a list to be complete. But the developer can expect that other application developers at the same organization will have considered the same set of risks.

The system specification generally is created from a set of requirements. The importance of solid system specification cannot be overemphasized. After all, without a specification, the behavior of a system cannot be wrong, it can only be surprising! Especially when it comes to running a business, no one wants security surprises.

A solid specification also draws a coherent big-picture view of what the system does and why. Specifications should be as formal as possible, without becoming overly arcane. Remember that the raison d'être for a specification is system understanding. In general, the clearer and easier to understand a specification is, the better the resulting system will be.

The importance of external analysis
Nobody builds a poor system intentionally. Developers are a proud lot, and for the most part they work hard to create solid working systems. This is precisely why a security risk analysis team should not include anyone from the design and development team. One essential way in which security testing differs from standard testing is in the importance of preserving a completely independent view of the system, divorced from design influences. When developers assume additional responsibilities as security testers, bad things are more likely to happen. Designers and developers often are too close to their systems, and skeptical that their systems might have flaws.

Therefore, it is better to get an outside team to perform security analysis and testing (often called a " tiger team" ). Doing so has the additional benefit of testing the integrity of the design documents of the system, since a good tiger team will exercise those documents extensively in their analysis. Of course, before such an analysis can be performed, the design team must make sure that the requirements and the program specification are so clear that an external team can completely understand the system.

An experienced team of external analysts considers many different scenarios during the course of an analysis. Examples of scenarios that are often considered include decompilation risks, eavesdropping attacks, playback attacks, and denial of service attacks. Testing is most effective when it is directed instead of random. Considering how these scenarios might be applied to your system can produce extremely relevant security tests.

There is another good reason for choosing an outside security analysis team: Even the best developers tend to lack the security expertise necessary to perform this sort of analysis well. Of course, the best results generally will be obtained by forming a team of high-priced external security gurus. Thankfully, such a team often is not necessary; it may be good enough to put together a team from your own organization made up of people with a good knowledge of security who were not involved in design decisions. Unfortunately, security expertise seems to be a rare commodity these days. Determining whether or not to seek help outside of your organization should be a decision based on the risk analysis: Are the resources at your disposal capable of mitigating sufficient risk?

Security testing versus functional testing
With a ranked set of potential risks in a system, testing for security is possible, though difficult. Testing is an empirical activity that requires a live system and close observation of that system. Security tests usually will not yield self-evident results such as obvious system penetrations. More often a system will behave in a strange or curious fashion that suggests to an analyst that something interesting is afoot. These sorts of hunches can be further explored through manual inspection of the code. For example, if an analyst can get a program to crash by feeding it really long inputs, a buffer overflow may exist -- and could potentially be leveraged into a security breach. The next step is to find out where and why the program crashed by looking at the source code.

Functional testing involves dynamically probing a system to determine whether the system behaves as intended under normal circumstances. Security testing, when well done, is different.

Security testing should involve probing a system in ways that an attacker might probe it, looking for weaknesses in the software that can be exploited. Security testing is most effective when it is directed by system risks that are unearthed during a risk analysis. This implies that security testing is a fundamentally creative form of testing that is only as strong as the risk analysis upon which it is based. Security testing is by nature bounded by identified risks -- as well as the security expertise of the tester.

Code coverage has proven to be a good metric for understanding how well a particular set of tests can uncover a system’s faults. It is always a good idea to use code coverage as a metric for measuring the effectiveness of functional testing. For security testing, code coverage plays an even more critical role. Simply put, if certain areas of a program (either functional or security) have never been exercised during testing, these areas should be immediately suspect in terms of security. One obvious risk is that unexercised code will include Trojan-horse vulnerability, whereby seemingly innocuous code carries out an attack. Less obvious (but more pervasive) is the risk that unexercised code has serious bugs that can be leveraged into a successful attack.

Dynamic testing versus static testing
Dynamic security testing can help ensure that such risks don't come back to bite you. Static analysis is useful as well. Many of today's security problems are echoes of well understood problems that can come around again. The fact that more than 50 percent of 1998's CERT alerts involved buffer overflow problems emphasizes this point. There is no reason why any code today should be susceptible to buffer overflow problems, yet they remain the biggest source-code security risk still.

It is possible to statically scan security-critical source code for known problems, fixing any problems encountered. Many hackers have code scanning tools that will scan through your code looking for potential problems, which they will later personally examine to determine whether a security problem does exist. The key to preventing such an approach is a deep knowledge of potential problems. Such tools are a good first step for lowering the bar of entry into the security analysis field, since they encode knowledge that would otherwise exist only in the heads of security experts, and they do so in a way that can be productive to all developers. It is therefore reasonable that developers themselves would use such a tool as an impartial third-party assessor.

Conclusion
Software must behave itself in myriad ways today. Good software assurance practices can help ensure that software behaves properly. Safety-critical and high-assurance systems have always made supreme efforts to analyze and track software behavior. Security-critical systems must follow suit. We can avoid the bandage-like penetrate-and-patch approach to security only if we consider security to be a crucial, complex, and integral system property, not a simple feature or an afterthought.

Computer security is increasingly important because the world is becoming highly interconnected and networks are being used to carry out critical transactions. Deciding to connect a local network to the Internet is a security-critical decision. The environment that machines must survive in has changed radically in recent years, and software security must anticipate those risks better than ever.

Software that fails in unexpected ways lies at the root of most security problems. Though software assurance has room to mature further, it offers a great deal to practitioners who want to strike at the heart of potential security problems.

Resources

• The first "Make your software behave" column on developerWorks, where Gary and John introduce their philosophy of security, and explain why they're focusing on software security issues facing developers
  
  
• The developerWorks columns by Gary and John have been updated and expanded in the book Building Secure Software, which also includes plenty of new material. Pick up a copy today -- your users will thank you.
  

Rate this article This content was helpful to me: Strongly disagree (1) Disagree (2) Neutral (3) Agree (4) Strongly agree (5) Comments?

developerWorks > Security >