Programmers today recognize, more than ever, their responsibility for producing well-written unit tests. Whether a developer uses test-driven development (TDD) or creates unit tests after writing the code, the evidence is clear that unit tests help produce high-quality, bug-free code.
Even though developers know that testing helps, we still see a reluctance by programmers to test all their code. They offer reasons ranging from lack of time, to ineffective tools, to problems writing tests with many dependent objects.
I'd like to look at unit testing and address these issues. Specifically, I want to offer some tips on how easy it can be to write unit tests with mock objects.
Let's look at the first two objections before we delve into mock objects.
One principle we all learn at an early age is that "things take time," especially if they are things worth doing. Few developers would dispute the value of unit testing, so we need to look at how we define "too long."
Developers are impatient; they want results. They like to code, run the code, and see results. In this respect, unit tests are made for them. A unit test provides the instant gratification developers seek, but many programmers think that the time spent writing the tests takes away from writing the application code, which is what they're paid to do. Certainly this is true if you just count the number of lines of application code (or some other measure) that the programmer produces in a specific period. But we have to look at the overall time spent per line of code shipped. If we stop calculating as soon as the code compiles and gets checked in, we might ignore the most costly part of creating software -- removing defects. The cost of defects goes up nonlinearly the further into the software development cycle defects are found. Many quality practices take a little more time up-front in the process, but reduce time significantly later on. This has been validated by many studies.
There are still too many programmers who think that it's someone else's job to find the errors in their code. While I have observed a dramatic improvement in this situation over the last decade, there are still too many programmers who don't take full responsibility for their work, nor do they take advantage of the tools and techniques that could help make their code better. I introduce unit testing to my students early in the software engineering course. I show them how to write tests using modern tools. I assign homework involving the writing of unit tests. Yet, when given the opportunity to adopt effective unit testing in their work, only about 25 percent of them do so. Why? They just don't feel that testing is important. Their intuition overrides their rationality. They have proof, but choose to ignore it.
Unit testing doesn't take long, but many programmers think it does. As an educator, I need to work hard to change this perception early in the students' careers and continue to reinforce it throughout their academic tenure. Commercial organizations must follow up by making unit testing a valued practice when they hire the graduates.
This is a poor excuse at best. There are many effective unit testing tools available to developers today. It doesn't matter what programming language you use or the other development tools you employ, there are unit testing tools available to you. Many of the tools are open source or free.
I use Eclipse as my primary programming environment. Of the unit testing tools that I have available in my current Eclipse configuration, the primary one I use is the JUnit test framework. Most Java programmers know about JUnit and have probably used it at least once. JUnit is an integral part of the Java developer tools in Eclipse. The platform makes it easy to create JUnit tests. I simply have to select a Java source file in the package browser and select New>JUnit Test Case from the context menu when I right-click on the selected file (see Figure 1). The support includes automatic creation of test methods for the class under test and much more. Running tests is just as easy as creating them. Eclipse has a separate view for viewing the results of your JUnit tests.
Figure 1: Creating a JUnit test in Eclipse
There are many more unit testing tools integrated with Eclipse. Many of them are based on JUnit and extend its capabilities. I have my students in the object-oriented and design class use the Coverlipse plug-in to measure the code coverage of their tests. I expect them to have 100% coverage in all of their application code. They don't like this at first, but by the middle of the term, their test coverage usually reaches 100%.
I have also loaded plug-ins for TestNG, djUnit, and the Eclipse Test and Performance Tools Platform (TPTP). Each of these has a set of features that support effective unit testing. The point is that there are many adequate unit testing tools available to every developer, so the lack of tools can no longer be used as an excuse for not producing unit tests.
A good unit test exercises a single method. In a well-designed system, objects work together to accomplish a task; therefore, in order to test a method, we often have to provide other objects that enable the method to complete its task. Objects in enterprise applications are complex, difficult to create, and depend upon external objects for their state. An application that uses a relational database has many such objects, like connections, statements, result sets, and so on. We want our unit tests to be simple and execute quickly. If we had to reset a database to a known state before each unit test, the tests would be quite complex and certainly would run slower than we desire.
A popular technique for simplifying unit tests is to create mock objects that are used just in the tests. A mock object is a replacement for a real object that we create for the express purpose of testing. Mock objects were championed by Tim Mackinnon, Steve Freeman, and Philip Craig1 and have become a staple in the unit testing toolbox. Several books and papers show how to use mock objects in unit testing. These describe the capabilities that mock objects should have and how to use them. Creating mock objects can, however, be quite difficult to create from scratch. We would like to have helpful automation for this task.
Some software tools, like EasyMock,2 provide automated help, but they can be complex and difficult to use as well. Further, they are not always compatible with our other development tools. But there are some easy ways to create mock objects, or equivalent capabilities, with existing tools. The rest of this article shows you some ways of doing this using the Eclipse platform.
Object-oriented design experts advise us to program to interfaces. When we do, the design is more robust, flexible, and responsive to change. We see an example of programming to interfaces when we use the Java JDBC™ API to work with databases. We'll consider a simple example based upon code in the JDBC API Tutorial and Reference, Second Edition. We have a table in the relational database that has columns called a, b, and c that have integer, string, and floating point values. The following method, in a class called DatabaseExample, takes a database Connection object, reads the records from the database, and prints the values.
In order to write a unit test for readABC( ), we need to have a Connector object, a Statement object, and a ResultSet object. Connector, Statement, and ResultSet are all interfaces declared in the java.sql package. Specific database systems provide concrete implementations of these interfaces. The readABC( ) method does not depend upon any specific database system. Our test should not depend on one, either. In fact, we may not have selected the database software yet. Since the method under test is written to the interfaces, we can create a mock object for each interface for our tests.
Let's start with the Connection object we pass into the method. The only Connection method we call is createStatement( ). But our Connection must implement all of the methods defined in the Connection interface. It's easy to create such an object using Eclipse. We perform the following steps:
- Create a new class in Eclipse (call it MockConection) that implements the Connection interface. Check the option to generate stubs for inherited abstract methods (see Figure 2). Eclipse generates the new class with default return values for non-void methods. If the return value is an object, the default is null.
- Find those methods that are called from the method under test and provide an implementation that behaves like the real object would.
- Add methods to the mock object that allow you to initialize it properly for your tests.
Figure 2: Creating a mock object class from an interface
In order to implement our version of the method, we need an object that implements the Statement interface. We will create a default one, called MockStatement, by following the first step above. Now we can implement our method with a simple change:
Since there's nothing else we need from our MockConnection class, we don't have to write any initialization code. We might need to add some later when we use the class for other tests (test code evolves, just like application code).
The MockStatement must provide behavior for executing a SQL query to the database. This means that it must return a ResultSet object. The ResultSet object has to provide the behavior used by our method under test. This is where we will use the third step in the process above. So let's create our MockResultSet object.
After we create the default object using the first step of our process, we need to provide an implementation for the next( ), getInt( ), getString( ), and getFloat( ) methods. Let's attack the next( ) method first. Next( ) simply advances the cursor to the next row returned from the query. This means that we need a collection in our MockResultSet that we can iterate over. We can do this by creating a private Collection instance in the MockResultSet and provide methods to access it and iterate over it. We begin by adding the following code to our class.
Now we have to create an instance of MockResultSet, populate it with the records that we want (expect) the database to return from the query, and connect our MockStatement object with the MockResultSet object.
For our first test, we want to make sure that we can return the results of one record. So we will create a MockResultSet object, populate it with the record, give it to a MockStatement, and give that MockStatement to the Connector. Our test case code looks like this:
First, we add a setStatement in the MockConnection. This simply saves a Statement (MockStatement) in the connection to return when the createStatement( ) method is invoked. When we execute the test, we get a null pointer exception at the beginning of the while loop in the readABC( ) method. The problem is that the executeQuery( ) method in our MockStatement doesn't return the MockResultSet. Let's change this.
I have adopted the convention to put in a comment "// MOCK" wherever I change the stub in a mock object to perform the appropriate actions to support my tests. Now when we run the test, it fails because the assertion expects the string we entered and gets back an empty string (not a null). We need to implement the next( ) method on our MockResultSet and then the methods to get the values from the record(s). The next( ) method implementation looks like this:
Finally, we implement the missing get...( ) methods. The following code shows how the getInt( ) method looks. The getString( ) and getFloat( ) methods are similar.
After we change the three methods, our test case runs and passes. We can now begin to add more tests and incrementally add code, as necessary, to our mock object.
At this point, it's worth asking what benefits we achieved with the mock objects and what did they cost. Let's consider the benefits first.
- We have written a unit test that exercises our application code. The application code uses a database, and we are able to run it without having a database.
- Our test code, specifically the mock objects, give us total control over the database results.
- The test runs quickly. There is no dependence upon actual database and network connections.
- The mock object classes we have can be reused and enhanced for future tests with minimal effort.
What did it cost us to implement the solution?
- We had to create three mock object classes. We did this automatically and did not have to write any code for the base implementations. If we had to implement the classes by coding stubs for each method, we would have written more than 140 stub methods for the MockResultSet alone!
- We had to write approximately 50 lines of code that actually implemented the behavior we needed in our mock object classes.
The benefits far outweigh the costs in my mind. The process for building mock objects from interfaces is simple, straightforward, and helps me develop effective unit test suites. This process works with most modern interactive development environments (IDEs), such as Eclipse.
Programming to interfaces is a good OO practice, but we are not always presented with legacy code that follows the practice. What then? What if the code we have uses a concrete class in its implementation? There are two ways to handle the situation. Both of these have approximately the same cost as the above solution.
If you only have a concrete class, you can create an interface from the class, change your application code to program to the interface, and then proceed as described above. This provides an extra benefit of improving your design by introducing more abstractions and programming to them, rather than the specific class. Refactoring support in Eclipse lets you create the interface with a single action. You select the class in the Package view, right-click, and select Refactor>Select Interface... . This method has slightly more cost in terms of effort to implement than when you start with an interface. Creating the interface is simple with most IDEs, but the number of changes in the application code can be significant if you decide to replace all source code uses of the concrete class with the interface.
This solution is not as desirable as creating an interface from the class. You create your subclass and override just those methods that you use in your methods under test. However, you must be careful that if you have a constructor that invokes the super class's constructor, that there are no hidden dependencies on external objects that you cannot control, such as databases, network connections, and so on. This is less desirable as the method above, because creating the interface does let you refactor your code to make it cleaner. This method is one that I would use when I do not have the ability to change the code under test. The total cost of this method is the same as using the interface to create a mock object class.
I hope that you see the benefits of using mock objects in unit testing. I also hope this brief tutorial helps you see how to create mock objects easily. At first, mock objects can seem quite daunting. Admittedly, some implementations are quite complex -- more so than they need be. However, the methods shown here are not the final word. Sometimes you may want to develop much more complex mock object classes because they will give you additional benefits. My advice is to start out simply, using the techniques shown here and then add complexity if you need it. Use other tools, or use pre-packaged mock objects. Some of these can be found on the mock objects' Webpages.4
There is one other potential solution that I'm currently exploring. I think this holds some hope and might lead to new developments in unit testing. This solution is the use of aspects for mocking the capabilities needed for your tests,5 but we'll leave that discussion for another day.
1 Tim Mackinnon, Steve Freeman, and Philip Craig, "Endo-testing: Unit testing with mock objects," in Extreme Programming Examined, Giancarlo Succi and Michele Marchesi ed., Addison-Wesley 2001.
3 White et al., JDBC API Tutorial and Reference, Second Edition. Addison Wesley 1999.
5 See Chapter 20, section 20.4 in AspectJ Cookbook, Russ Miles, O'Reilly 2005.
Gary Pollice is a professor of practice at Worcester Polytechnic Institute, in Worcester, MA. He teaches software engineering, design, testing, and other computer science courses, and also directs student projects. Before entering the academic world, he spent more than thirty-five years developing various kinds of software, from business applications to compilers and tools. His last industry job was with IBM Rational software, where he was known as "the RUP Curmudgeon" and was also a member of the original Rational Suite team. He is the primary author of Software Development for Small Teams: A RUP-Centric Approach, published by Addison-Wesley in 2004. He holds a BA in mathematics and an MS in computer science.