Staging adoption of EclEmma and Rational Team Concert integration

Increase unit testing quality with Code Coverage Advisor

Comments

There are many benefits of the integration between EclEmma and IBM Rational Team Concert, and the benefits affect the entire team. The integration helps teams to improve their levels of code coverage which improves the quality of code and reduces the number of defects. Because of this:

  • The development teams produce code with high levels of code coverage.
  • The testing team is free to focus on whether the code meets the business objectives, rather than getting blocked with many low-level code defects.

To understand the concepts in this tutorial, it helps to be familiar with the initial EclEmma and Rational Team Concert integration tool. For more information, read Improve the quality of your unit testing by integrating eclEmma and Rational Team Concert.

The Code Coverage Advisor is a Rational Team Concert process extension that prevents code from being delivered if there isn't sufficient code coverage, which helps teams to improve code quality. This tutorial describes the improvements in the new version of the integration tool. The Downloadable resources section contains the additional files needed to benefit from the recent enhancements to the tool.

When to use Code Coverage Advisor

Use Code Coverage Advisor:

  • For a new project to ensure that the unit tests provide sufficient code coverage. In this scenario, use the advisor to help the development team to do unit tests at the same rhythm as they do coding. This makes it easier to reach the highest level of code coverage.
  • For existing projects with poor or non-existent unit tests to help teams incrementally improve the unit testing and code coverage.

In both scenarios, Code Coverage Advisor helps teams improve the quality of their code by helping them to maintain high levels of code coverage from their unit tests. After you configure the code coverage and run the application, the Code Coverage Advisor looks at the coverage of the files being delivered to determine whether they have satisfactory code coverage. The code can't be delivered if the coverage is below the target level or if the code coverage is not enabled for the project. If this happens, the developer has to improve the coverage before delivering the code. You can set the advisor to check the coverage level of the new or modified lines only. When this check is set, only the newly inserted or modified lines of code are checked for code coverage. The percentage coverage of the new/modified code is compared to the threshold set in the team process.

Select Only check for new or modified lines to activate this setting:

Figure 1. Setting to activate new or modified lines only coverage
Screen capture showing the setting to activate a check for new or modified lines only
Screen capture showing the setting to activate a check for new or modified lines only

For example, if you modify the HelloWorld.java class, and it does not meet the coverage requirements, the Coverage Advisor tells you the percentage of coverage. In Figure 2, the coverage is 73.6%.

Figure 2. Code Coverage status
Screen capture showing the coverage requirements are not met
Screen capture showing the coverage requirements are not met

Figure 3 shows the coverage compared against the new or modified lines in delivery time.

Figure 3. Code coverage by new/modified lines
Screen capture showing an error in Team Advisor, coverage is insufficient
Screen capture showing an error in Team Advisor, coverage is insufficient

The coverage error in the Team Advisor view differs from the Coverage view because the Team Advisor view is based on the setting "Only check for new or modified lines." Adding new or modified lines to your testing path helps you reach the target coverage level.

Allow deliveries with insufficient code coverage

There are times when trusting that the developers are delivering "good quality" code is important in order to get the code ready, no matter the level of code coverage. This usually occurs in cases where it is more critical for the project to deliver the code before having unit testing in good shape. To allow developers to deliver code that does not meet the target level of code coverage, the process settings can be set such that the advisor:

  • Allows delivery, but tracks the code coverage level on the related work item
  • Allows the delivery of the code if the required approval is present

Tracking deliveries with insufficient code coverage

If the code has to be delivered regardless of the level of code coverage, the team lead developer can choose to allow deliveries and track the status in the associated work item to the changed set.

To do this, select Allow deliveries and only update associated workitems with Code Coverage status:

Figure 4. Allow deliveries with tracking option
Screen capture showing deliver and update associated workitems selected
Screen capture showing deliver and update associated workitems selected

When this option is selected, the delivery is allowed regardless of the level of the code coverage. However if the coverage is less than the required level, a comment and a code_coverage_failed tag are added to the associated work item:

Figure 5. Tracking messages in work item
Screen capture showing messages about code coverage
Screen capture showing messages about code coverage

This approach allows the team lead developer to quickly search for any work items with failing test coverage. The team lead developer can decide whether the coverage is acceptable, or whether more work needs to be scheduled in order to clean up the coverage. It also allows analysis at a later stage to see whether work items with less coverage have a higher density of defects.

Delivering code with insufficient code coverage

Even if the coverage level is under the low level, you can deliver code changes if the required approval is set. To do that, you need to have an approval with Ignore Code Coverage in each of the associated work items (see Figure 6). Make sure the "Allow Deliveries and Only update associated workitems with Code Coverage status" check box is not selected.

Figure 6. Ignore Code Coverage approval request
Screen capture showing Ignore Code Coverage approval details
Screen capture showing Ignore Code Coverage approval details

If the code coverage is insufficient and the approval isn’t in the required state, the advisor gives you an error:

Figure 7. Ignore Code Coverage error
Error shows there is no approval to ignore code coverage
Error shows there is no approval to ignore code coverage

Implement Code Coverage Advisor

To implement this approach in your own environment, install eclEmma from the Eclipse update site. The rest of the files you need are in the Downloadable resources section.

The code was developed on IBM Bluemix® DevOps Services in an open Rational Team Concert instance. If you want enhancements, or if you want to contribute to the advisor, contact the authors of this tutorial, or Steve Arnold, or open a work item at the eclEmma Rational Team Concert Code Coverage Advisor project.


Downloadable resources


Related topics


Comments

Sign in or register to add and subscribe to comments.

static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=Rational, DevOps
ArticleID=1026794
ArticleTitle=Staging adoption of EclEmma and Rational Team Concert integration
publish-date=02122016