Improve the quality of unit testing by increasing code coverage

Configure the Code Coverage Advisor in Rational Application Developer 8.0.3 and integrate with Rational Team Concert

One of the challenges for many agile software development teams is to ensure that their unit tests cover most of the code. This is critical in ensuring that they create as few defects as possible and that the code can be refactored. Therefore, one of the key metrics (other than all tests passing) is the amount of code that has been covered. Starting in version 8.0.3 of Rational Application Developer, you can configure IBM® Rational® Application Developer and integrate it with IBM® Rational Team Concert™ to run your tests and check code coverage before allowing the code to be delivered. This article walks you through how to set up this precondition, called the Code Coverage Advisor, and shows how you can use it to increase the coverage of code tested in your projects.

Steve Arnold (steve.arnold@uk.ibm.com), Senior Technical Consultant, IBM

photo of Steve ArnoldSteve lives in Twickenham, London with his wife and young daughter. He has worked as a technical consultant for IBM Rational software since 2000, where he is a certified scrum master specializing in agile project delivery, modeling, and patterns-based engineering. Outside of work, he enjoys spending time with his family and studying, teaching, and practicing Tai Chi. He has written several articles covering the latest features in Rational Software Architect 7.5 and 8, as well as articles discussing various extensions to Rational Software Architect, Rational RequisitePro, and Rational Team Concert.


developerWorks Professional author
        level

04 October 2011

Also available in Chinese

The measurement of how much code has been executed during development is a key metric in many kinds of development projects, regardless of whether you're using agile, Rational Unified Process (RUP), or waterfall development methods. This is important to both the developers and testers, because when developers produce code with high levels of code coverage, the testing team can focus on whether the code meets the business objectives, rather than getting blocked with lots of low-level code defects.

When the amount of code coverage is especially important

The following scenarios are common examples of when the amount of code coverage is important:

  • Any team that wants to be responsive to the business and make changes in a low-risk manner
  • Third parties who are developing code and who want to provide some visibility into the quality of their work or need to meet a contractual SLA
  • When a project defines a contract between development and testing for the amount of code coverage by development testing, which means less effort is spent by testing team on catching coding bugs
  • Agile teams that want to be able to make decision release code at end of every sprint (iteration), because that improves the quality of deliverables, so handing off the code before it is ready for production is less likely to be needed
  • Teams that want to reduce technical debt by incrementally increasing the number of unit tests and refactoring code to reduce errors and complexity
  • Organizations that want to establish metrics programs to measure quality and technical debt so that trends can be spotted and improvements made earlier

One of the specific challenges for many agile teams is to ensure that their unit tests are covering most of the code. This is critical in ensuring that developers create as few defects as possible and that the code can be refactored. (Given the trend in teams toward little or no design, it is vital this is supplemented with good quality unit tests.) One of the key metrics (other than the fact that all the tests are passing), is the amount of code that has been covered -and that's where the new version of IBM® Rational® Application Developer can help.

Earlier this year, I wrote a prototype extension for integrating IBM® Rational Team Concert™ and Rational Application Developer. It prevents code being delivered to Rational Team Concert if it doesn't have sufficient code coverage, as measured in Rational Application Developer. This extension, now called the Code Coverage Advisor, has been formally incorporated into Rational Application Developer and is available from version 8.0.3 onward. This means that you can now configure Rational Application Developer and IBM Rational Team Concert to check your code coverage before allowing the code to be delivered. This article walks you through how to set this up and shows you how it can be used to increase code coverage during development of your projects.

Tip:
You'll need some knowledge of Rational Application Developer to follow this article.

The sections that follow cover these three topics:

  1. When it might be appropriate to use the Code Coverage Advisor
  2. What happens when you try to deliver code with this extra process configured in Rational Application Developer
  3. How to configure Rational Team Concert to use the advisor

Situations where it's best to use the Code Coverage Advisor

So, how should you use the advisor? There are two major scenarios:

  • The first is for a new project, when you want to ensure that the unit tests provide sufficient code coverage. In that scenario, you can use the advisor to aid the development teams in maintaining high standards of unit testing and code coverage.
  • The second is for projects with poor or nonexistent unit tests that you want to improve. By switching on the advisor, the developers will be required to provide unit tests only for code that they modify, so those teams can start to improve in a simple, incremental, and manageable manner. Also, this approach means that the unit tests that are being written will cover the code that is changing; hence, they are most likely to uncover defects.

In both scenarios, the Code Coverage Advisor can provide a lot of value in helping teams improve the quality of their code by helping them to maintain high levels of code coverage from their unit tests. The next section explains how a developer delivers code using Rational Application Developer when the code coverage functionality is configured.


Delivering code with the new process in Rational Application Developer

The first thing that you need to do, if it hasn't been done already, is to configure your projects to collect code coverage information.

Configure the process in your projects

  1. Right-click on the project (or projects) in the Project Explorer, and select Properties.
  2. Then choose Code Coverage from the menu, and set up the code coverage information for the project in the same way that it is set up for the team (see Figure 1).
Figure 1. Project code coverage configuration
Configured with source file type and line coverage

What happens when you try to deliver code with the process enabled

After you have configured the code coverage, the Code Coverage Advisor will check the coverage for the files to determine whether the code can be delivered. The coverage advisor will not allow code to be delivered under these conditions:

  • Coverage is below the target level.
  • Coverage is out of date (for example, you have modified one of the files and not rerun the tests).
  • Code coverage is not enabled for the project.

In such cases, the developer will be required to improve the coverage before delivering the code. However, if the process is appropriately configured and the person is in a role that has necessary permissions, the developer can choose to override the warning and deliver the code with an unsatisfactory amount of code coverage.

Let's look at an example. In the project shown in Figure 2, you can see that the HelloWorld.java class is not meeting coverage requirements.

Figure 2. Project Explorer shows failing code coverage
HelloWorld.java is red, with 67% code coverage

If you try to deliver the code, the delivery will fail, and you will get the process advice shown in Figure 3: Overrule “Prohibit Unsatisfactory Code Coverage” precondition.

Figure 3. Code coverage delivery failure
Process warning that line coverage is insufficient

Setting up the process in Rational Team Concert

Storing the information in the process means that the required coverage percentage thresholds are stored centrally and apply equally across the team (although you can configure different coverage requirements for different components, as you will see later). You can set different coverage settings in your local client, but the server-based settings will be used when you deliver code.

Configure the process

Tip:
Before you start, you will need an appropriate role on the project that has permission to modify the project area's process.

  1. From the Eclipse shell, navigate to the Work Item perspective, and open the Team Artifacts view.
  2. Right-click on the project area that you want to configure, and choose Open.
  3. Then click the Process Configuration tab to open the process editor (Figure 4).
Figure 4. Process configuration editor
Roles, project, and team configuration options
  1. Open the Team Configuration view, and choose Operation Behavior.
  2. Scroll down the list of operations until you find the Deliver (client) operation, and select the cell under the Everyone (default) column, as shown in Figure 5.
Figure 5. Deliver operation
Deliver for everyone operation selected

Configure adequate code coverage as a precondition

You are now ready to configure the rule for checking that the code coverage is sufficient every time that someone delivers code.

  1. Add a precondition to check the code coverage:
    1. Click Add in the preconditions section, and choose Prohibit Unsatisfactory Code Coverage (see Figure 6).
    2. Configure the various code coverage levels that you require.

You can specify the required coverage at various different levels within the source code. For instance, you might require 70% line coverage for every file; however, the developer must have 100% coverage for every type. You can also provide regular expressions to exclude files from the coverage statistics, which is useful if you have some automatically generated code on which you don't care about coverage. Finally, you can have different rules for different components by specifying which components these coverage settings apply to.

Figure 6. Prohibit Unsatisfactory Code Coverage configuration
Dialog window for required coverage and component
  1. Now Save the process, and these new rules will apply the next time that you try to deliver code.

Resources

Learn

Get products and technologies

Discuss

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Select information in your profile (name, country/region, and company) is displayed to the public and will accompany any content you post. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into Rational software on developerWorks


static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=Rational, DevOps
ArticleID=762655
ArticleTitle=Improve the quality of unit testing by increasing code coverage
publish-date=10042011