from The Rational Edge: This article describes the practice of test-first programming, or TFP, which suggests creating tests for code before you actually write the code. Pollice explains the practice using an extended example and outlines its benefits for software developers and their teams.

Gary PolliceWorcester Polytechnic Institute

Gary Pollice is a professor of practice at Worcester Polytechnic Institute, in Worcester, Massachusetts. He teaches software engineering, design, testing, and other computer science courses, and also directs student projects. Before entering the academic world, he spent more than thirty-five years developing various kinds of software, from business applications to compilers and tools. His last industry job was with IBM Rational Software, where he was known as "the RUP Curmudgeon" and was also a member of the original Rational Suite team. He is the primary author of Software Development for Small Teams: A RUP-Centric Approach, published by Addison-Wesley in 2004. He holds a B.A. in mathematics and an M.S. in computer science.


developerWorks Master author
        level

15 June 2004

Illustration Last month I talked about the importance of matching project and process, urging you to consider the context in which you build software. This prompted me to think about whether software development has truly universal best practices. I think not. For any given practice, there is always at least one context in which it is not optimal. However, some practices are pretty darned good in most cases. One of these -- test-first programming, or TFP -- is the topic of this month's column.

Project scope and process flexibility

Before we launch into an examination of this practice, we need some background on when to use it. To understand this, let's look at the relationship between project scope and process flexibility. In my work with many clients over many years, I've observed that as the scope of a project expands, the need for a uniform, formal process increases proportionately (see Figure 1). Most experienced developers have a large collection of practices and techniques that they apply selectively, depending on the job. When they work alone on a task, they are able to do things their way. However, if they are on a team -- especially at the enterprise level -- the project manager must institute a higher level of process to keep work consistent across the entire team. Often this process must be relatively inflexible, as deviations from it can affect many people.

Figure 1: Relationship between project scope and process flexibility

Figure 1: Relationship between project scope and process flexibility

For example, let's say you are a software architect who has always described project functional requirements as a list of unrelated "shall have" items in a Software Requirement Specification (SRS). But then you attend a workshop on use cases and realize that they more accurately describe why systems will be valuable to stakeholders. Although you would unquestionably be adopting a better technique, switching to use cases on your current project would lead to chaos -- unless you can ensure that everyone on the team has made the switch as well.

Of course, if you are working on a small project within a larger one, you might be able to adopt different techniques to optimize your work, as long as they will not interfere with more widespread organizational practices. For example, if a large project requires specialized 2-D and 3-D graphics for its user interface, the subproject that implements the libraries for rendering these graphics might adopt special methods for describing requirements. This would have no negative impact on other process activities. Another subproject that is suitable for model-driven development might employ UML models to generate a significant portion of the subproject's code -- without interfering with the overall project's code generation procedures.


Test-First Programming: An individual practice for developers

When they have the flexibility to do things their way in a small project or sub-project, some developers choose to apply elements of the Extreme Programming (XP)1 approach. Although this article will not discuss what it means to "do XP" on a project, it will look at TFP, a practice associated with this methodology, which I believe helps many software professionals produce better work. Although you need discipline to apply and perfect it, I find that TFP is easy to understand and fits well with the way I work. It is also easy to demonstrate to my students, and I have taught them how to use it in term projects.

The explanation below is based on how I apply TFP. Although not an exhaustive description, it perhaps will provide enough information to whet your appetite and encourage you to learn more about TFP. I include several sources for this purpose in the References section.

A process for unit test implementation

TFP (also known as test-driven design or test-driven development) is actually a process for implementing the unit test practice that XP describes. Simply stated, the process is as follows: Before you write a single line of code, make sure you have a test that fails. In other words, write a test that exercises the code you are about to write.

Ron Jeffries describes the process as follows:

  • Find out what you have to do.
  • Write a unit test for the desired new capability. Pick the smallest increment of new capability you can think of.
  • Run the unit test. If it succeeds, you're done. Go to step 1, or if you are completely finished, go home.
  • Fix the immediate problem: maybe it's the fact that you didn't write the new method yet. Maybe the method doesn't quite work. Fix whatever it is. Go to step 3.2

This seems straightforward enough, but you must fulfill two requirements to adopt TFP successfully. First, you must know how to write a good test. Second, you must discipline yourself to write the tests before coding. This runs counter to everything you may have learned in your training.

What is a good test?

When deciding what to test, Jeffries says, "Pick the smallest increment of new capability you can think of." This is good advice. Most novices try to cram too many elements into each unit test. However, the smallest increment may not always be the right choice. As you become experienced at writing tests, you will find what is the most comfortable and efficient for you.

For me, a good test focuses on one thing. However, how I define that "one thing" may vary depending upon the type of code I am writing. This is similar to the principle for designing methods and classes. You want high cohesion throughout your design, and your tests should be cohesive as well.

Tools for TFP

To practice TFP you need automated tools. Basically, the TFP cycle is test/code/debug/refactor/repeat. Without tools that support rapid shifts from one activity to the next, you will quickly become frustrated and give up.

The typical supporting tools for TFP are good integrated development environments (IDEs) and unit test frameworks. I use the Eclipse IDE, with the JUnit plug-in, for all of my Java development.

JUnit is a unit testing framework designed by Kent Beck and Erich Gamma for testing Java programs.3 Variations of it support the testing of programs in other languages, such as cppUnit for C++. I used JUnit to create the example in the next section of this article.

An example using TFP

Classes and books on software testing often use a sample program that accepts three numbers and then performs tests for the type of triangle with side lengths equal to those numbers. If you test by the triangle's angles, possible types are right, acute, obtuse, or equiangular. If you test by lengths of the triangle's sides, possible types are equilateral, isosceles, or scalene. To demonstrate how to use TFP, we will create this sample program with a class called TriangleTester.4 In its purest form, TFP would require me to create the test class first, before I write a test. However, I find it easier to create the class file first, with some empty methods, and then create the test file. Eclipse's JUnit integration can generate the test class automatically, so I can minimize my work by letting the tools do it for me. Code Sample 1 shows the file TriangleTester.java that I created, including an empty method for the kindOfTriangle method.

Code Sample 1: Stub for kindOfTriangle method

Code Sample 1: Stub for kindOfTriangle method

At this point I must make a couple of decisions. First, I need to decide what type of number to use for arguments. I choose double-precision real numbers, which are the most general type. I also need to decide how to represent the triangle type. After some thought I create a separate type called TriangleType that indicates properties of the triangle.

Since there is no TriangleType class, I need to create it. This class, with just a constructor, is shown in Code Sample 2.

Code Sample 2: Empty TriangleType class

Code Sample 2: Empty TriangleType class

Now I'm ready to write the first test. Eclipse can create a test class for the TriangleTester class. By default it will be named TriangleTesterTest, but you can change the name to anything you want. Since I already have a method in my TriangleTester class (kindOfTriangle), I can have the Eclipse JUnit integration create a test method, testKindOfTriangle, that corresponds to it. Note that all JUnit test methods must begin with the word "test." Case is not important.

Many test cases are possible for testing kindOfTriangle. I need to decide if I will have just one test method or many test methods for the different test cases. My preference is to have many small test methods. So, instead of letting the tool create the initial test method, I will write each method as I go.

What should I test first? As I noted above, we can classify triangle types by angles or sides, and in some instances the three numbers we supply may not form a valid triangle. It doesn't matter which possibility I choose first, so I pick a right triangle.

I create the method called testRightTriangle and write a simple test, shown in Code Sample 3.

Code Sample 3: Right triangle test

Code Sample 3: Right triangle test

The Pythagorean Theorem tells us that a 3-4-5 triangle is a right triangle, so we can simply test for sides of 3, 4, and 5.5 Even after I import the junit.framework.Assert class, my test class will not compile. That is because the TriangleType class does not have a method called isRightTriangle; I need to create one.

When practicing TFP, you add only enough code to make your tests pass. In this case, I need a true value; the simplest way I can make this happen is by adding the code in Code Sample 4.

Code Sample 4: Initial isRightTriangle method

Code Sample 4: Initial isRightTriangle method

You might wonder whether I should write a test class for the TriangleType class. At this point I choose not to do so because everything I write in the TriangleType class will be tested in the TriangleTesterTest class.

Now I execute my first test. This is very simple in Eclipse. I select the TriangleTesterTest class and tell Eclipse to run that class as a JUnit test. When I run the test, it fails. Figure 1 shows the JUnit view I get in Eclipse. The red bar tells me that there was a failure, and the rest of the information in the view helps me determine what went wrong.

Figure 2: JUnit view of results from the first test run

Figure 2: JUnit view of results from the first test run

Click to enlarge

Ah, now I see the problem. I'm returning null from the kindOfTriangle method. I change the method to return an instance of TriangleType and rerun the test. Now the bar turns green, as shown in Figure 3.

Figure 3: JUnit view after modifying the code

Figure 3: JUnit view after modifying the code

What's next? What about triangles that are not right triangles? What if I put in different numbers and ask if they represent a right triangle? I add a test for that to the test method I've already written, as shown in Code Sample 5.

Code Sample 5: Adding a test for non-right triangles

Code Sample 5: Adding a test for non-right triangles

Now, when I run the test it fails at the assertion I just inserted. The reason is simple. I'm just returning true from the isRightTriangle method (Code Sample 4). I have to fix this before continuing with new functionality. Never add new code until the existing code passes all previous tests. First, I add code to check whether the triangle is a right triangle as shown in Code Sample 6.

Code Sample 6: Check for right triangle in main test

Code Sample 6: Check for right triangle in main test

Now, of course, I have to write the code for isRightTriangle and setRightTriangle. I add the method isRightTriangle to TriangleTester.java and setRightTriangle to TriangleType.java, as shown in Code Samples 7 and 8, respectively.

Code Sample 7: Code to check for right triangle

Code Sample 7: Code to check for right triangle

To keep the right triangle property I add a private variable and a setter method.

Code Sample 8: Maintaining the right triangle property

Code Sample 8: Maintaining the right triangle property

Now I get a green bar again in JUnit, so I can continue to add functionality. Should I go on to tests for other types of triangles? Perhaps, but at some point, I must deal with the fact that real number arithmetic is not exact on digital computers. I know this from experience. So now I'd like to see if the code I've written so far handles real numbers correctly. (I'm pretty sure that it doesn't, but I need a test to make sure.) I add the test in Code Sample 9 to my right triangle tests. Sure enough, JUnit displays a red bar: The test failed.

Code Sample 9: Test for real number precision

Code Sample 9: Test for real number precision

Now I have a design decision to make. How do I want to handle this precision problem? All solutions I can think of would require refactoring and rework. Experience tells me that some rework is inevitable. At least I now have a set of tests the code must pass, and I haven't gotten so far into coding that the amount of rework is excessive.6 I need a delta -- an acceptable amount for arithmetic error. I'd like to make my client tests remain as they are, so I add a default delta, remove the right triangle test return statement, and add the code shown in Code Sample 10 to TriangleTester.java.

Code Sample 10: Default precision delta

Code Sample 10: Default precision delta

Then, I continue to add tests, each time modifying the code by adding new code, changing existing code, and refactoring as I go. Eventually I have implemented a complete class in which I have confidence. I also have tests that I can run anytime. In the future, if I change the code, or someone else does, these tests can tell us if we have broken any existing functionality.

At this point, I'll stop giving you all of the details. Before moving on, however, let's look at Table 1, which shows the tests I wrote and the order in which I wrote them. We'll also look at some decisions I had to make as the coding progressed.

Table 1: Tests in order of creation
TestDescription
Positive right triangleCheck to make sure that a 3-4-5 triangle is recognized as a right triangle.
Negative right triangle Check to make sure that a 3-3-5 triangle is not recognized as a right triangle.
Real number precision Add a real number default delta to the program.
Variable default deltaAllow the client to change the default delta. This caused me to make the TriangleTester methods non-static, which required a change to the test class.
Test for delta as part of the method call Let the caller supply a delta value as part of the call to the kindOfTriangle method.
Test for invalid triangles Check to see if the values given to kindOfTriangle can actually represent a triangle. If they can't, return a special TriangleType.
Test to ensure NOT_A_TRIANGLE cannot be modifiedNOT_A_TRIANGLE is meant to be a constant, but it is not protected from change. To bulletproof the code that implements this special TriangleType, I created TriangleTypeTest.java.
Test combinations of sides on right trianglesMake sure that the program finds a right triangle, regardless of which side is the hypotenuse.
Isosceles triangle tests Make sure that an isosceles right triangle is recognized as both isosceles and right. This is similar to the right triangle test.
Equilateral triangle tests Similar to the previous tests.

There you have it. I implemented the TriangleTester and TriangleType classes with about 100 lines of Java code and 70 lines of test code.


Benefits of TFP

Adopting TFP can have several benefits, some of which I have experienced personally. Your work style and context will determine how much value you get from the practice.

Tests for your code

The most obvious benefit of TFP is that it gives you tests for the code you've written. It amazes me how many programmers today write code without bothering to test it. Perhaps some organizations do not clearly state their quality expectations for finished code, and many use a process that places the entire testing burden on the quality assurance group.

The fact is, everyone is responsible for quality -- regardless of how you define it. It is reasonable to expect that programmers will unit test their code before adding it to the rest of the project. If programmers use TFP, they have to test their code. There is no way of getting around it. Students who are just learning how to test code often protest that TFP does not necessarily result in good tests. I tell them they must decide whether "bad" tests are better than no tests at all. Perhaps we'll explore this issue in a future column.

The triangle tester example has a total of seven test methods, but it has twenty-three distinct tests. And this is not a complete set. I could add many more, but I'm confident that the tests I have developed are pretty solid. This is confidence I would not have if I had not written any tests.

High-percentage coverage

You can measure code coverage in several ways: by assessing line or statement coverage, condition coverage, branch coverage, and so on. When you adopt TFP, you can achieve 100 percent code coverage. Although my particular method for using TFP does not provide this (see the How much is enough? section below), it does provide an acceptable level of coverage for the code segments most likely to contain defects.

Testers agree that an acceptable level of code coverage is around 80 percent. Trying to achieve full coverage is often a waste of time; if you have to spend hours trying to force tests to exercise error conditions and exceptional cases, you get diminishing returns.

Theoretically, if you write code only to satisfy an existing test, then by definition you can achieve coverage for every line of code you write. However, this may not be so in practice. I am not aware of any empirical studies that demonstrate a 100 percent coverage benefit.

The important point is that adopting TFP guarantees that you will achieve a significant amount of code coverage in your unit tests -- probably far more than what you achieve today.

Test-driven design

Designs evolve. We know this to be true from experience. When business conditions change or stakeholders develop new requirements, we need ways to modify existing code. That is why software should be soft and flexible.

When you use TFP to drive your design, in effect you have adopted a test-driven design (TDD) approach to building software that naturally leads to simpler, more flexible architectures.

In the triangle tester example above, I revised my design based upon the tests I wrote. I added the constant NOT_A_TRIANGLE object in response to my test. Based on the final result, I might decide to remove that constant and add a field to the TriangleType class that indicates an invalid triangle. The design will evolve.

As I implemented the tests for equilateral and isosceles triangles, I realized that I was using only the delta value for calculations in right triangles. Perhaps I will want to add a "fuzz factor" with deltas for determining whether I have an isosceles or equilateral triangle, respectively. However, since I don't need that now, I can put it off to another day -- when I might really need it.

TFP/TDD is a practice that I will use to supplement my other design tools and techniques. As I gain more experience, I will find more ways of applying it and understand better when and where it is appropriate.

Tests = specs

When you finish using TFP, the tests you have created represent a set of executable specifications. As long as you keep your tests and code synchronized (and that's the whole point of the practice), if a programmer wants to know what the system does, he or she can look at the tests.

Tests are not the only type of specification you need. It is not reasonable to ask clients to look at the tests to see if your specifications are correct. As software engineers, we know there are many types of requirements that we can represent in different ways. It is foolish to think that one type of requirement will satisfy every stakeholder or that it will be easy to keep requirements synchronized with the code. However, TFP does provide support in this area.

Working in inch pebbles

Professor Mike Ciaraldi, one of my colleagues at Worcester Polytechnic Institute talks about measuring project progress in "inch pebbles" rather than milestones. We work on a very small increment, get it right, and then move on to the next increment. Eventually, all of our inch pebbles add up to a milestone.

If we use tests to bound our increments, we can get immediate satisfaction by writing our first test and then implementing code to make it work. I have found that adopting TFP helps prevent analysis paralysis, that condition that makes us afraid to write code because we don't yet know all of the constraints and possible problems we might face.

As the old Chinese proverb says, "A journey of a thousand miles begins with the first step." TFP helps us take that first step, and then the next, and so on.

Easy to learn

TFP is an easy practice to learn. I introduce it in one hour-long class during the term, providing references and walking students through an example such as the triangle tester program. At the end of this hour, they are ready to go out and try their luck.

Several students take to TFP quickly and report that it has changed the way they create programs. Others are uncomfortable with it, and I don't force them to adopt the practice. For now, I want them to experience many software development practices and use the ones that match their style. However, if somewhere along their career paths they need TFP, at least they will know how to use it. If you are a project manager, you might want to use a similar approach and spend a modest amount of time training your team to use TFP.


How much is enough?

Most programmers who want to adopt TFP will ask: "How much is enough? Do I have to write tests for every line of code and every function"?

There are different opinions on this. I take the middle ground and test most of the methods I implement. However, I do not bother writing tests for methods that the IDE generates for me.

How much testing you do depends upon your project goals and available time. Although it might be nice to create extensive tests for every method you write, you must decide how much you can actually do and whether the return will warrant the time you put in.


A versatile practice

TFP is a useful practice, whether or not you use it as a design tool. It provides extensive coverage and is a relatively easy way to prevent coding disasters. If we think the code we're about to write is simple, we tend to cut corners. But if you've ever seen a system fail because of one line change, then you can appreciate the need to test as much as possible.

You can apply TFP within almost any project context, whether or not the organization or project process dictates its use. When they see the quality of your code, maybe your teammates will ask what you've done to improve it. You can easily share your "secret" and try to make it part of the team's process. Even if you don't succeed at this, you can still be proud of the tests and code you deliver.


References

  • http://www.junit.org/index.htm: the home page for the JUnit project offers code and articles about how to apply TFP. Be sure to read "Test Infected -- Programmers Love Writing Tests," an article in the documentation section.
  • http://c2.com/cgi/wiki?CodeUnitTestFirst: the XP Wiki Web page devoted to TFP.
  • Kent Beck, Test Driven Development by Example. Addison-Wesley, 2002. Describes Beck's approach to TDD.
  • David Astels, Test-Driven Development: A Practical Guide. Prentice Hall, 2003. Provides learning via examples.
  • Andrew Hunt and David Thomas, Pragmatic Unit Testing in Java with JUnit. The Pragmatic Programmers, LLC, 2003. Instruction for experienced TFP users.

Click here to download the sample Triangle Tester program.


Notes

1 Readers unfamiliar with XP can consult http://c2.com/cgi/wiki?ExtremeProgrammingRoadmap.

2 See http://c2.com/cgi/wiki?CodeUnitTestFirst.

3 For information about JUnit, visit http://www.junit.org. Read the Test Infected paper at this site for a good overview of the framework and the process. Also, the References section at the end of my article lists some of the many books and papers that describe how to use JUnit in the context of TFP.

4 We won't show a complete program, just a class that will do the job. It's easy to test this class with Eclipse and JUnit. You can download all the code for the class by clicking on the link at the end of this article.

5 The Pythagorean Theorem is 32 + 42 = 9 + 16 = 25 = 52.

6 Clearly, if I had waited until later in the development cycle to add this test, I'd have to do more rework. Experience counts.

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Select information in your profile (name, country/region, and company) is displayed to the public and will accompany any content you post. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into Rational software on developerWorks


static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=Rational
ArticleID=87317
ArticleTitle=Test before you code
publish-date=06152004