Last month I talked about the importance of matching project and process, urging you to consider the context in which you build software. This prompted me to think about whether software development has truly universal best practices. I think not. For any given practice, there is always at least one context in which it is not optimal. However, some practices are pretty darned good in most cases. One of these -- test-first programming, or TFP -- is the topic of this month's column.
Project scope and process flexibility
Before we launch into an examination of this practice, we need some background on when to use it. To understand this, let's look at the relationship between project scope and process flexibility. In my work with many clients over many years, I've observed that as the scope of a project expands, the need for a uniform, formal process increases proportionately (see Figure 1). Most experienced developers have a large collection of practices and techniques that they apply selectively, depending on the job. When they work alone on a task, they are able to do things their way. However, if they are on a team -- especially at the enterprise level -- the project manager must institute a higher level of process to keep work consistent across the entire team. Often this process must be relatively inflexible, as deviations from it can affect many people.
Figure 1: Relationship between project scope and process flexibility
For example, let's say you are a software architect who has always described project functional requirements as a list of unrelated "shall have" items in a Software Requirement Specification (SRS). But then you attend a workshop on use cases and realize that they more accurately describe why systems will be valuable to stakeholders. Although you would unquestionably be adopting a better technique, switching to use cases on your current project would lead to chaos -- unless you can ensure that everyone on the team has made the switch as well.
Of course, if you are working on a small project within a larger one, you might be able to adopt different techniques to optimize your work, as long as they will not interfere with more widespread organizational practices. For example, if a large project requires specialized 2-D and 3-D graphics for its user interface, the subproject that implements the libraries for rendering these graphics might adopt special methods for describing requirements. This would have no negative impact on other process activities. Another subproject that is suitable for model-driven development might employ UML models to generate a significant portion of the subproject's code -- without interfering with the overall project's code generation procedures.
Test-First Programming: An individual practice for developers
When they have the flexibility to do things their way in a small project or sub-project, some developers choose to apply elements of the Extreme Programming (XP)1 approach. Although this article will not discuss what it means to "do XP" on a project, it will look at TFP, a practice associated with this methodology, which I believe helps many software professionals produce better work. Although you need discipline to apply and perfect it, I find that TFP is easy to understand and fits well with the way I work. It is also easy to demonstrate to my students, and I have taught them how to use it in term projects.
The explanation below is based on how I apply TFP. Although not an exhaustive description, it perhaps will provide enough information to whet your appetite and encourage you to learn more about TFP. I include several sources for this purpose in the References section.
A process for unit test implementation
TFP (also known as test-driven design or test-driven development) is actually a process for implementing the unit test practice that XP describes. Simply stated, the process is as follows: Before you write a single line of code, make sure you have a test that fails. In other words, write a test that exercises the code you are about to write.
Ron Jeffries describes the process as follows:
- Find out what you have to do.
- Write a unit test for the desired new capability. Pick the smallest increment of new capability you can think of.
- Run the unit test. If it succeeds, you're done. Go to step 1, or if you are completely finished, go home.
- Fix the immediate problem: maybe it's the fact that you didn't write the new method yet. Maybe the method doesn't quite work. Fix whatever it is. Go to step 3.2
This seems straightforward enough, but you must fulfill two requirements to adopt TFP successfully. First, you must know how to write a good test. Second, you must discipline yourself to write the tests before coding. This runs counter to everything you may have learned in your training.
What is a good test?
When deciding what to test, Jeffries says, "Pick the smallest increment of new capability you can think of." This is good advice. Most novices try to cram too many elements into each unit test. However, the smallest increment may not always be the right choice. As you become experienced at writing tests, you will find what is the most comfortable and efficient for you.
For me, a good test focuses on one thing. However, how I define that "one thing" may vary depending upon the type of code I am writing. This is similar to the principle for designing methods and classes. You want high cohesion throughout your design, and your tests should be cohesive as well.
Tools for TFP
To practice TFP you need automated tools. Basically, the TFP cycle is test/code/debug/refactor/repeat. Without tools that support rapid shifts from one activity to the next, you will quickly become frustrated and give up.
The typical supporting tools for TFP are good integrated development environments (IDEs) and unit test frameworks. I use the Eclipse IDE, with the JUnit plug-in, for all of my Java development.
JUnit is a unit testing framework designed by Kent Beck and Erich Gamma for testing Java programs.3 Variations of it support the testing of programs in other languages, such as cppUnit for C++. I used JUnit to create the example in the next section of this article.
An example using TFP
Classes and books on software testing often use a sample program that accepts
three numbers and then performs tests for the type of triangle with side lengths
equal to those numbers. If you test by the triangle's angles, possible
types are right, acute, obtuse, or equiangular. If you test by lengths of the
triangle's sides, possible types are equilateral, isosceles, or scalene.
To demonstrate how to use TFP, we will create this sample program with a class
TriangleTester.4 In its purest form, TFP would require me to create the
test class first, before I write a test. However, I find it easier to create
the class file first, with some empty methods, and then create the test file.
Eclipse's JUnit integration can generate the test class automatically,
so I can minimize my work by letting the tools do it for me. Code Sample 1 shows
the file TriangleTester.java that I created, including an empty method for the
Code Sample 1: Stub for kindOfTriangle method
At this point I must make a couple of decisions. First, I need to decide what
type of number to use for arguments. I choose double-precision real numbers,
which are the most general type. I also need to decide how to represent the
triangle type. After some thought I create a separate type called
that indicates properties of the triangle.
Since there is no
TriangleType class, I need to create it. This class, with
just a constructor, is shown in Code Sample 2.
Code Sample 2: Empty TriangleType class
Now I'm ready to write the first test. Eclipse can create a test class
TriangleTester class. By default it will be named
but you can change the name to anything you want. Since I already have a method
TriangleTester class (
kindOfTriangle), I can have the Eclipse JUnit integration
create a test method,
testKindOfTriangle, that corresponds to it. Note that all
JUnit test methods must begin with the word "test." Case is not
Many test cases are possible for testing
kindOfTriangle. I need to decide if
I will have just one test method or many test methods for the different test
cases. My preference is to have many small test methods. So, instead of letting
the tool create the initial test method, I will write each method as I go.
What should I test first? As I noted above, we can classify triangle types by angles or sides, and in some instances the three numbers we supply may not form a valid triangle. It doesn't matter which possibility I choose first, so I pick a right triangle.
I create the method called
testRightTriangle and write a simple test, shown
in Code Sample 3.
Code Sample 3: Right triangle test
The Pythagorean Theorem tells us that a 3-4-5 triangle is a right triangle,
so we can simply test for sides of 3, 4, and 5.5 Even after I import the
class, my test class will not compile. That is because the
does not have a method called
isRightTriangle; I need to create one.
When practicing TFP, you add only enough code to make your tests pass. In this case, I need a true value; the simplest way I can make this happen is by adding the code in Code Sample 4.
Code Sample 4: Initial isRightTriangle method
You might wonder whether I should write a test class for the
At this point I choose not to do so because everything I write in the
TriangleType class will be tested in the
Now I execute my first test. This is very simple in Eclipse. I select the
TriangleTesterTest class and tell Eclipse to run that class as a JUnit test.
When I run the test, it fails. Figure 1 shows the JUnit view I get in Eclipse.
The red bar tells me that there was a failure, and the rest of the information
in the view helps me determine what went wrong.
Figure 2: JUnit view of results from the first test run
Ah, now I see the problem. I'm returning null from the
method. I change the method to return an instance of
TriangleType and rerun
the test. Now the bar turns green, as shown in Figure 3.
Figure 3: JUnit view after modifying the code
What's next? What about triangles that are not right triangles? What if I put in different numbers and ask if they represent a right triangle? I add a test for that to the test method I've already written, as shown in Code Sample 5.
Code Sample 5: Adding a test for non-right triangles
Now, when I run the test it fails at the assertion I just inserted. The reason
is simple. I'm just returning
true from the
isRightTriangle method (Code
Sample 4). I have to fix this before continuing with new functionality. Never
add new code until the existing code passes all previous tests. First, I add
code to check whether the triangle is a right triangle as shown in Code Sample
Code Sample 6: Check for right triangle in main test
Now, of course, I have to write the code for
I add the method
TriangleType.java, as shown in Code Samples 7 and 8, respectively.
Code Sample 7: Code to check for right triangle
To keep the right triangle property I add a private variable and a setter method.
Code Sample 8: Maintaining the right triangle property
Now I get a green bar again in JUnit, so I can continue to add functionality. Should I go on to tests for other types of triangles? Perhaps, but at some point, I must deal with the fact that real number arithmetic is not exact on digital computers. I know this from experience. So now I'd like to see if the code I've written so far handles real numbers correctly. (I'm pretty sure that it doesn't, but I need a test to make sure.) I add the test in Code Sample 9 to my right triangle tests. Sure enough, JUnit displays a red bar: The test failed.
Code Sample 9: Test for real number precision
Now I have a design decision to make. How do I want to handle this precision
problem? All solutions I can think of would require refactoring and rework.
Experience tells me that some rework is inevitable. At least I now have a set
of tests the code must pass, and I haven't gotten so far into coding that
the amount of rework is excessive.6 I need a
delta -- an acceptable amount for arithmetic error. I'd like to
make my client tests remain as they are, so I add a default delta, remove the
right triangle test return statement, and add the code shown in Code Sample
Code Sample 10: Default precision delta
Then, I continue to add tests, each time modifying the code by adding new code, changing existing code, and refactoring as I go. Eventually I have implemented a complete class in which I have confidence. I also have tests that I can run anytime. In the future, if I change the code, or someone else does, these tests can tell us if we have broken any existing functionality.
At this point, I'll stop giving you all of the details. Before moving on, however, let's look at Table 1, which shows the tests I wrote and the order in which I wrote them. We'll also look at some decisions I had to make as the coding progressed.
Table 1: Tests in order of creation
|Positive right triangle||Check to make sure that a 3-4-5 triangle is recognized as a right triangle.|
|Negative right triangle||Check to make sure that a 3-3-5 triangle is not recognized as a right triangle.|
|Real number precision||Add a real number default delta to the program.|
|Variable default delta||Allow the client to change the default delta. This caused me to
make the |
|Test for delta as part of the method call||Let the caller supply a delta value as part of the call to the |
|Test for invalid triangles||Check to see if the values given to |
| Test to ensure |
|Test combinations of sides on right triangles||Make sure that the program finds a right triangle, regardless of which side is the hypotenuse.|
|Isosceles triangle tests||Make sure that an isosceles right triangle is recognized as both isosceles and right. This is similar to the right triangle test.|
|Equilateral triangle tests||Similar to the previous tests.|
There you have it. I implemented the
with about 100 lines of Java code and 70 lines of test code.
Benefits of TFP
Adopting TFP can have several benefits, some of which I have experienced personally. Your work style and context will determine how much value you get from the practice.
Tests for your code
The most obvious benefit of TFP is that it gives you tests for the code you've written. It amazes me how many programmers today write code without bothering to test it. Perhaps some organizations do not clearly state their quality expectations for finished code, and many use a process that places the entire testing burden on the quality assurance group.
The fact is, everyone is responsible for quality -- regardless of how you define it. It is reasonable to expect that programmers will unit test their code before adding it to the rest of the project. If programmers use TFP, they have to test their code. There is no way of getting around it. Students who are just learning how to test code often protest that TFP does not necessarily result in good tests. I tell them they must decide whether "bad" tests are better than no tests at all. Perhaps we'll explore this issue in a future column.
The triangle tester example has a total of seven test methods, but it has twenty-three distinct tests. And this is not a complete set. I could add many more, but I'm confident that the tests I have developed are pretty solid. This is confidence I would not have if I had not written any tests.
You can measure code coverage in several ways: by assessing line or statement coverage, condition coverage, branch coverage, and so on. When you adopt TFP, you can achieve 100 percent code coverage. Although my particular method for using TFP does not provide this (see the How much is enough? section below), it does provide an acceptable level of coverage for the code segments most likely to contain defects.
Testers agree that an acceptable level of code coverage is around 80 percent. Trying to achieve full coverage is often a waste of time; if you have to spend hours trying to force tests to exercise error conditions and exceptional cases, you get diminishing returns.
Theoretically, if you write code only to satisfy an existing test, then by definition you can achieve coverage for every line of code you write. However, this may not be so in practice. I am not aware of any empirical studies that demonstrate a 100 percent coverage benefit.
The important point is that adopting TFP guarantees that you will achieve a significant amount of code coverage in your unit tests -- probably far more than what you achieve today.
Designs evolve. We know this to be true from experience. When business conditions change or stakeholders develop new requirements, we need ways to modify existing code. That is why software should be soft and flexible.
When you use TFP to drive your design, in effect you have adopted a test-driven design (TDD) approach to building software that naturally leads to simpler, more flexible architectures.
In the triangle tester example above, I revised my design based upon the tests
I wrote. I added the constant
NOT_A_TRIANGLE object in response to my test.
Based on the final result, I might decide to remove that constant and add a
field to the
TriangleType class that indicates an invalid triangle. The design
As I implemented the tests for equilateral and isosceles triangles, I realized that I was using only the delta value for calculations in right triangles. Perhaps I will want to add a "fuzz factor" with deltas for determining whether I have an isosceles or equilateral triangle, respectively. However, since I don't need that now, I can put it off to another day -- when I might really need it.
TFP/TDD is a practice that I will use to supplement my other design tools and techniques. As I gain more experience, I will find more ways of applying it and understand better when and where it is appropriate.
Tests = specs
When you finish using TFP, the tests you have created represent a set of executable specifications. As long as you keep your tests and code synchronized (and that's the whole point of the practice), if a programmer wants to know what the system does, he or she can look at the tests.
Tests are not the only type of specification you need. It is not reasonable to ask clients to look at the tests to see if your specifications are correct. As software engineers, we know there are many types of requirements that we can represent in different ways. It is foolish to think that one type of requirement will satisfy every stakeholder or that it will be easy to keep requirements synchronized with the code. However, TFP does provide support in this area.
Working in inch pebbles
Professor Mike Ciaraldi, one of my colleagues at Worcester Polytechnic Institute talks about measuring project progress in "inch pebbles" rather than milestones. We work on a very small increment, get it right, and then move on to the next increment. Eventually, all of our inch pebbles add up to a milestone.
If we use tests to bound our increments, we can get immediate satisfaction by writing our first test and then implementing code to make it work. I have found that adopting TFP helps prevent analysis paralysis, that condition that makes us afraid to write code because we don't yet know all of the constraints and possible problems we might face.
As the old Chinese proverb says, "A journey of a thousand miles begins with the first step." TFP helps us take that first step, and then the next, and so on.
Easy to learn
TFP is an easy practice to learn. I introduce it in one hour-long class during the term, providing references and walking students through an example such as the triangle tester program. At the end of this hour, they are ready to go out and try their luck.
Several students take to TFP quickly and report that it has changed the way they create programs. Others are uncomfortable with it, and I don't force them to adopt the practice. For now, I want them to experience many software development practices and use the ones that match their style. However, if somewhere along their career paths they need TFP, at least they will know how to use it. If you are a project manager, you might want to use a similar approach and spend a modest amount of time training your team to use TFP.
How much is enough?
Most programmers who want to adopt TFP will ask: "How much is enough? Do I have to write tests for every line of code and every function"?
There are different opinions on this. I take the middle ground and test most of the methods I implement. However, I do not bother writing tests for methods that the IDE generates for me.
How much testing you do depends upon your project goals and available time. Although it might be nice to create extensive tests for every method you write, you must decide how much you can actually do and whether the return will warrant the time you put in.
A versatile practice
TFP is a useful practice, whether or not you use it as a design tool. It provides extensive coverage and is a relatively easy way to prevent coding disasters. If we think the code we're about to write is simple, we tend to cut corners. But if you've ever seen a system fail because of one line change, then you can appreciate the need to test as much as possible.
You can apply TFP within almost any project context, whether or not the organization or project process dictates its use. When they see the quality of your code, maybe your teammates will ask what you've done to improve it. You can easily share your "secret" and try to make it part of the team's process. Even if you don't succeed at this, you can still be proud of the tests and code you deliver.
- http://www.junit.org/index.htm: the home page for the JUnit project offers code and articles about how to apply TFP. Be sure to read "Test Infected -- Programmers Love Writing Tests," an article in the documentation section.
- http://c2.com/cgi/wiki?CodeUnitTestFirst: the XP Wiki Web page devoted to TFP.
- Kent Beck, Test Driven Development by Example. Addison-Wesley, 2002. Describes Beck's approach to TDD.
- David Astels, Test-Driven Development: A Practical Guide. Prentice Hall, 2003. Provides learning via examples.
- Andrew Hunt and David Thomas, Pragmatic Unit Testing in Java with JUnit. The Pragmatic Programmers, LLC, 2003. Instruction for experienced TFP users.
Click here to download the sample Triangle Tester program.
1 Readers unfamiliar with XP can consult http://c2.com/cgi/wiki?ExtremeProgrammingRoadmap.
3 For information about JUnit, visit http://www.junit.org. Read the Test Infected paper at this site for a good overview of the framework and the process. Also, the References section at the end of my article lists some of the many books and papers that describe how to use JUnit in the context of TFP.
4 We won't show a complete program, just a class that will do the job. It's easy to test this class with Eclipse and JUnit. You can download all the code for the class by clicking on the link at the end of this article.
5 The Pythagorean Theorem is 32 + 42 = 9 + 16 = 25 = 52.
6 Clearly, if I had waited until later in the development cycle to add this test, I'd have to do more rework. Experience counts.