Improve the quality of your unit testing by integrating eclEmma and Rational Team Concert

This article describes how to set up eclEmma and IBM Rational Team Concert so that you can automatically check your code coverage when code is delivered to a stream. The delivery can then be prevented if the coverage is not sufficient. This process makes it possible for teams to ensure that their unit tests are being effective and that they have high levels of coverage. It also helps teams with existing code to incrementally improve the coverage and quality of the code.

Share:

Steve Arnold (steve.arnold@uk.ibm.com), Certified IT Specialist, IBM

photo of Steve ArnoldSteve lives in Twickenham, London, with his wife and 5-year-old daughter. Professionally he has worked as a technical consultant for IBM Rational since 2000, where he is a certified scrum master specializing in agile project delivery, modeling, and patterns-based engineering. Outside of work he enjoys spending time with his family, studying, teaching, and practicing Taichi. He has written several articles that cover the latest features in IBM Rational Software Architect 7.5 and 8, and articles that describe various extensions to Rational Software Architect, IBM Rational RequisitePro, and IBM Rational Team Concert.


developerWorks Professional author
        level

Ioan Jones (ioan.jones@gmail.com), Development Architect

Author1 photoIoan enjoys finding creative solutions to interesting problems.  His professional focus is designing and implementing systems that support all phases of the software development lifecycle and its various methodologies.  This has afforded him experience with a variety of programming languages and technologies.  As a young software engineer leading his first major project, he realized the importance of the user experience, which led to a fundamental change in his approach to software development and an M.S. in Human-Computer Interaction.  A licensed Commercial Pilot, Ioan has a passion for anything that flies.  He also enjoys running, putting in miles for health and the occasional marathon.



08 April 2014

Introduction

The initial section of this article describes the benefits of the integration between eclEmma, a widely used, open-source, Java™ code coverage tool for Eclipse, and IBM® Rational Team Concert™. This integration helps teams improve their levels of code coverage, which improves the quality of code and reduces the number of defects. The article assumes no knowledge of eclEmma or Rational Team Concert and is suitable for anyone who is interested in ways to improve code quality. The latter sections of the article walk through the setup and configuration of the integration, and assume a familiarity with both tools.

The measurement of how much code has been run during development is a key metric in many kinds of development projects (including agile, IBM Rational Unified Process, and waterfall). It's important to both the developers and testers, because when development teams produce code with high levels of code coverage, the testing team is free to focus on whether the code meets the business objectives, rather than getting blocked with many low-level code defects.

The following situations are examples of when the amount of code coverage is important:

  • When a team wants to be responsive to the business and make changes in a low-risk manner
  • When third parties who are developing code want to provide some visibility into the quality of their work or need to measure quality to meet a contractual service-level agreement (SLA)
  • When contracts between teams (for example, development and test teams) make it possible for the test team to spend less effort on catching coding bugs
  • When teams want to reduce technical debt by incrementally increasing the number of unit tests and refactoring code to reduce errors and complexity
  • When organizations want to establish metrics programs to measure quality and technical debt to help spot trends and make improvements

Agile teams need to ensure that their unit tests cover most of the code. Significant code coverage ensures that as few defects as possible are created, and that the code is refactorable. (Given the agile trend towards little or no design, agile development teams need to supplement the development process with strong unit tests). One of the key metrics is the amount of code that has been run by unit tests. This article describes how to use Rational Team Concert and eclEmma to help measure and improve code coverage.

The Code Coverage Advisor described in this article provides a Rational Team Concert process extension that prevents code from being delivered if it doesn't have sufficient code coverage. The required level of code coverage is defined by the team and is measured using the values held by eclEmma in the client. You can configure Rational Team Concert to check your code coverage before it allows the code to be delivered. This article describes how to set up this process and shows you how to use it to increase code coverage in your projects.

The article consists of three sections:


Choose when to use the Code Coverage Advisor

The advisor can be used in two major scenarios:

  • For a new project to ensure that the unit tests provide sufficient code coverage. In this scenario, you can use the advisor to help the development teams to maintain high standards of unit testing and code coverage.
  • For existing projects with poor or non-existent unit tests to help them incrementally improve the unit testing and code coverage. By switching on the advisor, the developers will only be required to provide unit tests for code they modify. In this way, the teams can start to improve in a simple, incremental, and manageable manner. With this approach, unit tests that are being written will cover the code that is changing, and are most likely to uncover defects.

In both scenarios, the advisor can help teams improve the quality of their code by helping them to maintain high levels of code coverage from their unit tests. The next step is to deliver code using Rational Team Concert when the code coverage functionality is configured.


Deliver code with the new process and Rational Team Concert

The first step is to configure your projects to collect code coverage information. On the toolbar, click the eclEmma icon icon as shown in Figure 1, and choose Coverage Configurations....

Figure 1. Press the eclEmma icon and choose Coverage Configurations
Configure eclEmma code coverage

As shown in Figure 2, the Coverage Configurations dialog opens. It is similar to the Run/Debug Configurations dialog, but the Coverage Configurations dialog has a Coverage tab. After you choose which elements of code you want to capture coverage for, click Coverage to run your code and gather coverage information.

Figure 2. Coverage tab in the Coverage Configurations dialog
List of analysis scope areas on Coverage tab

After you configure the code coverage and run the application, the Code Coverage Advisor looks at the coverage for the files that are being delivered to determine whether they have satisfactory code coverage. The advisor does not allow code to be delivered if the coverage is below the target level or if the code coverage is not enabled for the project. In that case, the developer is required to improve the coverage before delivering the code, or the developer might be able to override the rule if the override option was enabled when the Code Coverage Advisor was set up.

For example, if you modify the HelloWorld.java class, and it doesn't meet the coverage requirements set by the team, when the code is delivered, the developer sees a process failure, as shown in Figure 3.

Figure 3. Code coverage delivery failure
Warning shows where line coverage is insufficient

Set up the process

The coverage thresholds are stored in the Rational Team Concert process, which means that the required coverage thresholds are stored centrally and enforced equally across the team. However, you can configure different coverage requirements for different components. Before you start, you need a role that has permission to modify the project area's process.

From the Eclipse shell, navigate to the Work Item perspective and open the Team Artifacts view. Right-click the project area you want to configure and click Open. Open the Process Configuration tab to open up the process editor, as shown in Figure 4.

Figure 4. Process Configuration Editor
Roles, project, and team configuration

Select Team Configuration> Operation Behavior. Scroll down the list of operations to the Deliver (client) operation, and select the box in the Everyone (default) column, as shown in Figure 5.

Figure 5. Deliver (client) operation
Selected deliver for Everyone operation

You are now ready to configure the rule for checking that the code coverage is sufficient, every time someone delivers code.


Configure the code coverage precondition

Add a precondition to check the code coverage. As shown in Figure 6, click Add in the preconditions section and choose Prohibit Unsatisfactory Code Coverage (EclEmma). On the left side of the Preconditions dialog, you can configure the various code coverage levels you require. For example, you might require 70% line coverage for every file. However, the developer needs to have 100% coverage for every type.

You can also provide regular expressions to exclude files from the coverage statistics. This is useful if you have some automatically generated code for which you don't need to ensure code coverage. You can also specify which components the coverage file applies to. With this approach, you can have different rules for different components.

Figure 6. Prohibit unsatisfactory code coverage configuration
Required coverage and component

Save the process. These new rules will apply the next time you try to deliver code.


Download and install the components

To implement this approach in your own environment, you need two main components, which can both be installed using Eclipse update sites. You can install eclEmma and you can install the Code Coverage Advisor from the Eclipse update site supplied with this article.

The code was developed on DevOps Services (JazzHub) in an open Rational Team Concert instance. If you want enhancements or if you want to contribute to the advisor, contact the author or raise some work items at the eclEmma Rational Team Concert Code Coverage Advisor project.


Downloads

DescriptionNameSize
Code coverage advisor com.eclemma.rtcclientadvisor.update_site_1_0_0_0.zip 49KB

Download

DescriptionNameSize
Code coverage advisorcom.eclemma.rtcclientadvisor.update_site_1_0_0_049KB

Resources

Learn

Get products and technologies

  • Download Rational Team Concert from Jazz.net and try it free on up to 10 developers for as long as you want (requires registration). If you'd prefer, you can try it in the sandbox instead, without installing it on your own system.
  • Evaluate IBM software in the way that suits you best: Download it for a trial, try it online, use it in a cloud environment, or spend a few hours in the SOA Sandbox learning how to implement service-oriented architecture efficiently.

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into DevOps on developerWorks


static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=DevOps, Rational
ArticleID=967768
ArticleTitle=Improve the quality of your unit testing by integrating eclEmma and Rational Team Concert
publish-date=04082014