Improve the quality of unit testing by increasing code coverage
Configure the Code Coverage Advisor in Rational Application Developer 8.0.3 and integrate with Rational Team Concert
The measurement of how much code has been executed during development is a key metric in many kinds of development projects, regardless of whether you're using agile, Rational Unified Process (RUP), or waterfall development methods. This is important to both the developers and testers, because when developers produce code with high levels of code coverage, the testing team can focus on whether the code meets the business objectives, rather than getting blocked with lots of low-level code defects.
When the amount of code coverage is especially important
The following scenarios are common examples of when the amount of code coverage is important:
- Any team that wants to be responsive to the business and make changes in a low-risk manner
- Third parties who are developing code and who want to provide some visibility into the quality of their work or need to meet a contractual SLA
- When a project defines a contract between development and testing for the amount of code coverage by development testing, which means less effort is spent by testing team on catching coding bugs
- Agile teams that want to be able to make decision release code at end of every sprint (iteration), because that improves the quality of deliverables, so handing off the code before it is ready for production is less likely to be needed
- Teams that want to reduce technical debt by incrementally increasing the number of unit tests and refactoring code to reduce errors and complexity
- Organizations that want to establish metrics programs to measure quality and technical debt so that trends can be spotted and improvements made earlier
One of the specific challenges for many agile teams is to ensure that their unit tests are covering most of the code. This is critical in ensuring that developers create as few defects as possible and that the code can be refactored. (Given the trend in teams toward little or no design, it is vital this is supplemented with good quality unit tests.) One of the key metrics (other than the fact that all the tests are passing), is the amount of code that has been covered -and that's where the new version of IBM® Rational® Application Developer can help.
Earlier this year, I wrote a prototype extension for integrating IBM® Rational Team Concert™ and Rational Application Developer. It prevents code being delivered to Rational Team Concert if it doesn't have sufficient code coverage, as measured in Rational Application Developer. This extension, now called the Code Coverage Advisor, has been formally incorporated into Rational Application Developer and is available from version 8.0.3 onward. This means that you can now configure Rational Application Developer and IBM Rational Team Concert to check your code coverage before allowing the code to be delivered. This article walks you through how to set this up and shows you how it can be used to increase code coverage during development of your projects.
You'll need some knowledge of Rational Application Developer to follow this article.
The sections that follow cover these three topics:
- When it might be appropriate to use the Code Coverage Advisor
- What happens when you try to deliver code with this extra process configured in Rational Application Developer
- How to configure Rational Team Concert to use the advisor
Situations where it's best to use the Code Coverage Advisor
So, how should you use the advisor? There are two major scenarios:
- The first is for a new project, when you want to ensure that the unit tests provide sufficient code coverage. In that scenario, you can use the advisor to aid the development teams in maintaining high standards of unit testing and code coverage.
- The second is for projects with poor or nonexistent unit tests that you want to improve. By switching on the advisor, the developers will be required to provide unit tests only for code that they modify, so those teams can start to improve in a simple, incremental, and manageable manner. Also, this approach means that the unit tests that are being written will cover the code that is changing; hence, they are most likely to uncover defects.
In both scenarios, the Code Coverage Advisor can provide a lot of value in helping teams improve the quality of their code by helping them to maintain high levels of code coverage from their unit tests. The next section explains how a developer delivers code using Rational Application Developer when the code coverage functionality is configured.
Delivering code with the new process in Rational Application Developer
The first thing that you need to do, if it hasn't been done already, is to configure your projects to collect code coverage information.
Configure the process in your projects
- Right-click on the project (or projects) in the Project Explorer, and select Properties.
- Then choose Code Coverage from the menu, and set up the code coverage information for the project in the same way that it is set up for the team (see Figure 1).
Figure 1. Project code coverage configuration
What happens when you try to deliver code with the process enabled
After you have configured the code coverage, the Code Coverage Advisor will check the coverage for the files to determine whether the code can be delivered. The coverage advisor will not allow code to be delivered under these conditions:
- Coverage is below the target level.
- Coverage is out of date (for example, you have modified one of the files and not rerun the tests).
- Code coverage is not enabled for the project.
In such cases, the developer will be required to improve the coverage before delivering the code. However, if the process is appropriately configured and the person is in a role that has necessary permissions, the developer can choose to override the warning and deliver the code with an unsatisfactory amount of code coverage.
Let's look at an example. In the project shown in Figure 2, you can see that the HelloWorld.java class is not meeting coverage requirements.
Figure 2. Project Explorer shows failing code coverage
If you try to deliver the code, the delivery will fail, and you will get the process advice shown in Figure 3: Overrule “Prohibit Unsatisfactory Code Coverage” precondition.
Figure 3. Code coverage delivery failure
Setting up the process in Rational Team Concert
Storing the information in the process means that the required coverage percentage thresholds are stored centrally and apply equally across the team (although you can configure different coverage requirements for different components, as you will see later). You can set different coverage settings in your local client, but the server-based settings will be used when you deliver code.
Configure the process
Before you start, you will need an appropriate role on the project that has permission to modify the project area's process.
- From the Eclipse shell, navigate to the Work Item perspective, and open the Team Artifacts view.
- Right-click on the project area that you want to configure, and choose Open.
- Then click the Process Configuration tab to open the process editor (Figure 4).
Figure 4. Process configuration editor
- Open the Team Configuration view, and choose Operation Behavior.
- Scroll down the list of operations until you find the Deliver (client) operation, and select the cell under the Everyone (default) column, as shown in Figure 5.
Figure 5. Deliver operation
Configure adequate code coverage as a precondition
You are now ready to configure the rule for checking that the code coverage is sufficient every time that someone delivers code.
- Add a precondition to check the code coverage:
- Click Add in the preconditions section, and choose Prohibit Unsatisfactory Code Coverage (see Figure 6).
- Configure the various code coverage levels that you require.
You can specify the required coverage at various different levels within the source code. For instance, you might require 70% line coverage for every file; however, the developer must have 100% coverage for every type. You can also provide regular expressions to exclude files from the coverage statistics, which is useful if you have some automatically generated code on which you don't care about coverage. Finally, you can have different rules for different components by specifying which components these coverage settings apply to.
Figure 6. Prohibit Unsatisfactory Code Coverage configuration
- Now Save the process, and these new rules will apply the next time that you try to deliver code.
- Find out more about Rational Application Developer:
- What's new in Rational Application Developer Version 8.0 on the Rational Application Developer wiki.
- This viewlet shows how to use the Code Coverage Advisor to prevent delivery of code when a predefined code coverage threshold is not met.
- Browse the Rational Application Developer for WebSphere Software page on developerWorks for links to technical articles and many related resources.
- Explore the user documentation in the Information Center.
- Learn more about Rational Team Concert:
- Find Rational Team Concert articles and links to many other resources on IBM developerWorks.
- Check the Rational Team Concert page on Jazz.net.
- Watch the Using Rational Team Concert in a globally distributed team webcast or a demonstration of the Dashboards and reports, or listen to the podcast about IBM Rational Team Concert and Jazz.
- Read Leaner software development with the aid of Collaborative Lifecycle Management, an article by Paula White and Steve Arnold (developerWorks, April 2011).
- Improve your skills. Check the Rational training and certification catalog, which includes many types of courses on a wide range of topics. You can take some of them anywhere, any time, and many of the “Getting Started” ones are free.
- Get the Rational Team Concert trial downloads (free):
- Evaluate IBM software in the way that suits you best.