The initial section of this article describes the benefits of the integration between eclEmma, a widely used, open-source, Java™ code coverage tool for Eclipse, and IBM® Rational Team Concert™. This integration helps teams improve their levels of code coverage, which improves the quality of code and reduces the number of defects. The article assumes no knowledge of eclEmma or Rational Team Concert and is suitable for anyone who is interested in ways to improve code quality. The latter sections of the article walk through the setup and configuration of the integration, and assume a familiarity with both tools.
The measurement of how much code has been run during development is a key metric in many kinds of development projects (including agile, IBM Rational Unified Process, and waterfall). It's important to both the developers and testers, because when development teams produce code with high levels of code coverage, the testing team is free to focus on whether the code meets the business objectives, rather than getting blocked with many low-level code defects.
The following situations are examples of when the amount of code coverage is important:
- When a team wants to be responsive to the business and make changes in a low-risk manner
- When third parties who are developing code want to provide some visibility into the quality of their work or need to measure quality to meet a contractual service-level agreement (SLA)
- When contracts between teams (for example, development and test teams) make it possible for the test team to spend less effort on catching coding bugs
- When teams want to reduce technical debt by incrementally increasing the number of unit tests and refactoring code to reduce errors and complexity
- When organizations want to establish metrics programs to measure quality and technical debt to help spot trends and make improvements
Agile teams need to ensure that their unit tests cover most of the code. Significant code coverage ensures that as few defects as possible are created, and that the code is refactorable. (Given the agile trend towards little or no design, agile development teams need to supplement the development process with strong unit tests). One of the key metrics is the amount of code that has been run by unit tests. This article describes how to use Rational Team Concert and eclEmma to help measure and improve code coverage.
The Code Coverage Advisor described in this article provides a Rational Team Concert process extension that prevents code from being delivered if it doesn't have sufficient code coverage. The required level of code coverage is defined by the team and is measured using the values held by eclEmma in the client. You can configure Rational Team Concert to check your code coverage before it allows the code to be delivered. This article describes how to set up this process and shows you how to use it to increase code coverage in your projects.
The article consists of three sections:
- When is it appropriate to use the Code Coverage Advisor?
- What happens when you try to deliver code with the extra process configured?
- How do you set up the process to enable the code coverage advisor?
Choose when to use the Code Coverage Advisor
The advisor can be used in two major scenarios:
- For a new project to ensure that the unit tests provide sufficient code coverage. In this scenario, you can use the advisor to help the development teams to maintain high standards of unit testing and code coverage.
- For existing projects with poor or non-existent unit tests to help them incrementally improve the unit testing and code coverage. By switching on the advisor, the developers will only be required to provide unit tests for code they modify. In this way, the teams can start to improve in a simple, incremental, and manageable manner. With this approach, unit tests that are being written will cover the code that is changing, and are most likely to uncover defects.
In both scenarios, the advisor can help teams improve the quality of their code by helping them to maintain high levels of code coverage from their unit tests. The next step is to deliver code using Rational Team Concert when the code coverage functionality is configured.
Deliver code with the new process and Rational Team Concert
The first step is to configure your projects to collect code coverage information. On the toolbar, click the eclEmma icon as shown in Figure 1, and choose Coverage Configurations....
Figure 1. Press the eclEmma icon and choose Coverage Configurations
As shown in Figure 2, the Coverage Configurations dialog opens. It is similar to the Run/Debug Configurations dialog, but the Coverage Configurations dialog has a Coverage tab. After you choose which elements of code you want to capture coverage for, click Coverage to run your code and gather coverage information.
Figure 2. Coverage tab in the Coverage Configurations dialog
After you configure the code coverage and run the application, the Code Coverage Advisor looks at the coverage for the files that are being delivered to determine whether they have satisfactory code coverage. The advisor does not allow code to be delivered if the coverage is below the target level or if the code coverage is not enabled for the project. In that case, the developer is required to improve the coverage before delivering the code, or the developer might be able to override the rule if the override option was enabled when the Code Coverage Advisor was set up.
For example, if you modify the HelloWorld.java class, and it doesn't meet the coverage requirements set by the team, when the code is delivered, the developer sees a process failure, as shown in Figure 3.
Figure 3. Code coverage delivery failure
Set up the process
The coverage thresholds are stored in the Rational Team Concert process, which means that the required coverage thresholds are stored centrally and enforced equally across the team. However, you can configure different coverage requirements for different components. Before you start, you need a role that has permission to modify the project area's process.
From the Eclipse shell, navigate to the Work Item perspective and open the Team Artifacts view. Right-click the project area you want to configure and click Open. Open the Process Configuration tab to open up the process editor, as shown in Figure 4.
Figure 4. Process Configuration Editor
Select Team Configuration> Operation Behavior. Scroll down the list of operations to the Deliver (client) operation, and select the box in the Everyone (default) column, as shown in Figure 5.
Figure 5. Deliver (client) operation
You are now ready to configure the rule for checking that the code coverage is sufficient, every time someone delivers code.
Configure the code coverage precondition
Add a precondition to check the code coverage. As shown in Figure 6, click Add in the preconditions section and choose Prohibit Unsatisfactory Code Coverage (EclEmma). On the left side of the Preconditions dialog, you can configure the various code coverage levels you require. For example, you might require 70% line coverage for every file. However, the developer needs to have 100% coverage for every type.
You can also provide regular expressions to exclude files from the coverage statistics. This is useful if you have some automatically generated code for which you don't need to ensure code coverage. You can also specify which components the coverage file applies to. With this approach, you can have different rules for different components.
Figure 6. Prohibit unsatisfactory code coverage configuration
Save the process. These new rules will apply the next time you try to deliver code.
Download and install the components
To implement this approach in your own environment, you need two main components, which can both be installed using Eclipse update sites. You can install eclEmma and you can install the Code Coverage Advisor from the Eclipse update site supplied with this article.
The code was developed on DevOps Services (JazzHub) in an open Rational Team Concert instance. If you want enhancements or if you want to contribute to the advisor, contact the author or raise some work items at the eclEmma Rational Team Concert Code Coverage Advisor project.
|Code coverage advisor||com.eclemma.rtcclientadvisor.update_site_1_0_0_0.zip||49KB|
- More about Rational Team Concert:
- "Improve the quality of unit testing by increasing code coverage with RTC and Rational Application Developer" (developerWorks, October 2011) explains how to configure the Code Coverage Advisor in Rational Application Developer and integrate with Rational Team Concert.
- Find Rational Team Concert articles and links to many other resources on IBM developerWorks, and check the product overview page, features and benefits, system requirements, and the user information center.
- Check the Rational Team Concert page on Jazz.net.
- Watch the Using Rational Team Concert in a globally distributed team webcast or a demonstration of the Dashboards and reports, or listen to the podcast about IBM Rational Team Concert and Jazz.
- Get more information about Java code coverage for Eclipse with eclEmma.
- Why use JazzHub?Discover the top 5 ways JazzHub can benefit your business.
- Explore the Rational software area on developerWorks for technical resources, best practices, and information about Rational collaborative and integrated solutions for software and systems delivery.
- Watch developerWorks demos, ranging from product installation and setup demos for beginners to advanced functionality for experienced developers.
- Improve your skills. Check the Rational training and certification catalog, which includes many types of courses on a wide range of topics. You can take some of them anywhere, anytime, and many of the Getting Started ones are free.
Get products and technologies
- Download Rational Team Concert from Jazz.net and try it free on up to 10 developers for as long as you want (requires registration). If you'd prefer, you can try it in the sandbox instead, without installing it on your own system.
- Evaluate IBM software in the way that suits you best: Download it for a trial or try it online.