IBM® Rational® Developer for i Version 9.1 includes a new code coverage feature. Code coverage tools help you run a code coverage session on an IBM i batch program that generates a report, which shows the percentage of lines in your code that have been covered by the current run and gives line by line coverage details. Code coverage supports the common IBM i compiled languages such as RPG, COBOL, C/C++, and CL. It also supports the generation of code coverage reports in three different formats: workbench report, HTML report, and PDF report.
Code coverage tools are also available on IBM® Rational® Developer for System z® and IBM® Rational® Developer for AIX® and Linux®. This article uses RPG programs to describe the code coverage features on IBM i.
Starting a code coverage session
You can start a code coverage session using one of the following solutions:
- Use Code Coverage As > Batch to start code coverage with the default parameters
- Use Code Coverage (Prompt) > Batch to open the launch configuration dialog and modify the code coverage parameters
- Launch from an existing shortcut under the Code Coverage toolbar
Preparing the program for code coverage
Code coverage tools are implemented using debug technology. You can run
code coverage on any program that you can debug at the source level. To
run code coverage on an IBM i program, you need to compile it using one of
Programs compiled with the
DBGVIEW(*LIST) option are not
supported by code coverage in RDi version 9.1.
Launching code coverage with one step
You can run code coverage on an IBM i program through the Remote System Explorer perspective. Select Code Coverage As > Batch to run a code coverage session using the default parameters without a prompt.
From the Remote Systems navigator, right-click the program you want to run code coverage on, and then select Code Coverage As > Batch to start a new code coverage session. The status of the code coverage job is displayed in the status bar at the lower right corner of the IDE. When the code coverage run finishes, a new workbench report is opened in the current perspective. The Compiled Code Coverage Launch History view is also opened, with a new launch added to the view, as shown in Figure 1.
Figure 1. Code coverage in action
Customizing code coverage parameters from the prompt
You can also launch a new code coverage session from the Compiled Code Coverage launch configuration dialog box, to modify the code coverage parameters for the current run. To open the dialog box, right-click an IBM i program and select Code Coverage (Prompt) > Batch.
Figure 2. Compiled code coverage prompt
From the What To Analyze tab, you can add another program or service program into the code coverage run by selecting Add.... In the current code coverage driver, if an IBM i program invokes another program or service program during its run, the secondary programs or service programs are not automatically included in the final report. You need to manually add them to the code coverage run so that they will appear in the final report.
From the How To Start tab, you can modify the parameters of your IBM i program. You can also select Prompt... to bring up an SBMJOB dialog box to modify the job settings interactively. Use this approach when you need to add parameters to the program call or when the program is launched by another, such as a CL setup program.
From the Code Coverage tab, you can select a code coverage level (Line, Procedure, or Program). You can also customize the report generation options, or specify to use the previous result feature. See "Code coverage collection options" for how to use these features.
Launching a code coverage session from a shortcut
After a code coverage run, a new shortcut is added to the Code Coverage toolbar drop-down list in the main toolbar. Select the shortcut to run code coverage on the same program using the same parameters of the last run. From the Code Coverage toolbar drop-down list, you can also select Compiled Code Coverage Configurations... to open the launch configuration dialog and modify the parameters of an existing code coverage configuration.
Using the code coverage UI
The code coverage UI provides many features for you to visualize and manipulate a code coverage report. For example, from the opened workbench report, you can navigate through all code coverage elements in a tree structure and view their coverage statistics. You can open the source editor and see the line-by-line coverage details at the annotation column. You can also generate an HTML report or a PDF report for a code coverage launch.
Working with the workbench report
A workbench report is opened in the editor area when the code coverage session finishes without an error. The code coverage elements are organized in a Program > Module > Source File > Procedure tree structure in the report. You can expand each element to display its children.
In the workbench report, the Coverage column shows the coverage percentage
for each element. The Covered column shows the number of lines that are
covered for each element and the Total column shows the total number of
lines for each element. As an example, in Figure 1, the
finish_tc procedure has a total number of 9 lines, and 7 of
them are covered in the current code coverage run, which results in a
coverage rate of 78 percent.
You can click on a column header to sort the coverage results by that column. For example, clicking on the Coverage column sorts the elements by their coverage percentage. To hide or show specific elements in the report, click the filter icon at the top right corner of a workbench report and select a filter from the drop-down list.
Viewing code coverage annotations in the editor
From a workbench report, double-click a source file or a procedure to open a source editor with coverage annotations, as shown in Figure 3. A green bar at the annotation column denotes a code region that is covered in the current run and a red bar denotes an uncovered code region. Hover over each annotation for a detailed text description. The colors for code coverage elements are configurable. From the Preferences page, click General > Appearance > Colors and Fonts to configure the colors of the covered and uncovered elements under the Code Coverage node.
Figure 3. Code coverage annotations in editor
If the source editor is opened on a source file, the pointer is placed at
the beginning of the file. If it is opened on a procedure, the pointer is
placed at the first line of the procedure. The editor in Figure 3 is
opened by double clicking the
finish_tc procedure from the
Setting code coverage acceptance levels
You can specify code coverage acceptance levels for one or more of the following elements: program, module, source file, or procedure. If the coverage percentage of a code coverage element is lower than the specified threshold for that particular element type, the code coverage report is considered a failure. All criteria must be met for the program to be flagged as passing the code coverage requirements.
To set these thresholds, in the Compiled Code Coverage Launch History view, click the down arrow at the top right corner and select Preferences... to open the preference dialog box. By default, a program threshold of 80 percent is used as the only acceptance criterion.
The icon at the first column of the Compiled Code Coverage Launch History view indicates whether the acceptance criteria passed or failed. Additionally, each failed element is decorated in the workbench report view with a red "x" overlay.
Note: If a child element fails to meet the thresholds, all of its ancestors are marked as failed.
Figure 4 shows one passed and one failed report in the Compiled Code Coverage Launch History view, with different indicator icons. Some procedures in the workbench report are decorated with a red "x" overlay when the procedure threshold is set to 90 percent.
Figure 4. Acceptance indicators in report view and Launch History view
Organizing code coverage results in Launch History view
A launch is added to the Compiled Code Coverage Launch History view for each code coverage run. You can manage the existing launches from the Launch History view. For example, you can delete one or more launches or rename a launch. To invoke an action on a launch, right-click it and select a menu item from the context menu.
To save all of the files for a code coverage run into a ZIP file, select the Export... action. To import the result back into the workspace, select Import... action. The Import feature supports importing from remote systems.
A code coverage run generates a few data files that are saved on the local file system. If you are curious about the data files generated by the code coverage run or if you want to copy them to a different location, select the Properties... action, which opens the properties dialog for the selected launch and shows the paths of the data files and the values of other code coverage properties.
Comparing two code coverage runs
As shown in Figure 5, you can generate a comparison report, which displays the differences between the two code coverage runs from two existing launches. Select two launches from the Compiled Code Coverage Launch History view and invoke the Compare coverage results action from the context menu.
In the comparison report, the number outside the parentheses is for the most recent run and the number inside the parentheses shows the change in that metric (either in terms of the coverage percentage or the number of lines covered) between the most recent and the previous launch.
Figure 5. Comparison report
Working with HTML reports
In addition to the workbench report, code coverage tools can also generate an HTML report, which displays the coverage statistics in a set of HTML pages. Figure 6 shows a sample HTML report.
Figure 6. HTML report
To generate an HTML report, right-click on a launch in the Compiled Code Coverage Launch History view and select Generate Report... and then select HTML report from the opened dialog box. If the HTML report is already generated, select the Open > HTML report action from the context menu to open an existing HTML report.
An HTML report contains three navigation panes on the left side (All Programs, All Modules and All Source Files), which help you easily navigate through the code coverage elements. An Overview Summary page is initially displayed on the right pane. You can select a link from a pane to display the coverage information about the selected element. If the code coverage element is a source file, the right pane displays the list of all procedures and their coverage statistics for that source file. If the source of the selected file is available, the source and the code coverage annotations are also displayed after the procedure listing.
Code coverage tools also support the generation of PDF report, which is a good format for printing.
Code coverage collection options
To customize the way a report is generated, select the Code Coverage tab from the Compiled Code Coverage launch configuration dialog box. By default, code coverage generates reports at the line level, as shown in Figure 7, which includes line-by-line statistics of code coverage information. If you only want to know whether a procedure is hit or not, set the code coverage level to Procedure. If you care only about whether a program is hit in the current run, set the code coverage level to Program. Higher granular code coverage level usually takes a shorter time to run, because it has to gather less information for the run.
Figure 7. Collection options in Code Coverage tab
To generate HTML or PDF reports after data collection finishes, in the Report Options group, select Generate HTML report or Generate PDF report. You can later use the Open > HTML report or Open > PDF report action to open these reports without having to generate them first using the Generate Report... action.
Code coverage supports the use of a previous result file, which provides the following benefits:
- Faster collection because it doesn't have to track lines or procedures that are already covered.
- A merged result by appending the new result to the previous result. This is especially valuable when running multiple test cases on the same program. The result produced at the end of the last test case includes an overall code coverage for all the test cases. It produces a measure of the effectiveness of your test suite.
To use the previous result feature, select either Append to the previous result or Append to a specific result in the Code Coverage Results group. The first option automatically locates the most recent result for the same program and uses that as the previous result. For the second option, you have to specify the path of the previous result file you want to use.
The code coverage UI is built on Eclipse (an open source platform) and benefits from an advanced help system. For example, hovering over a button produces a tool tip, and clicking F1 brings up the context sensitive help for the current view or the currently selected menu action. You can also select Help > Help Contents from the main menu to open the product help page and navigate to Developing > Determining Code Coverage for compiled language applications for help on code coverage.
The integrated code coverage tools in Rational Developer for i provide rich problem determination capabilities. The tools support a variety of compiled languages and report types on the IBM i platform. The report view gives you see a quick summary of all the code coverage elements and their coverage statistics and the integrated editor makes it possible to visualize the line-by-line coverage details from the source editor.
- Learn more about Rational Developer for i.
- What these YouTube demos of different Rational IDE scenarios.
- Explore the Rational software development on developerWorks for technical resources, best practices, and information about Rational collaborative and integrated solutions for software and systems delivery.
- Stay current with developerWorks technical events and webcasts focused on a variety of IBM products and IT industry topics.
- Improve your skills. Check the Rational training and certification catalog, which includes many types of courses on a wide range of topics. You can take some of them anywhere, anytime, and many of the Getting Started ones are free.
Get products and technologies
- Download the free trial of Rational Developer for i.
- Download a free trial version of other Rational software.
- Evaluate other IBM software in the way that suits you best: Download it for a trial, try it online, or use it in a cloud environment.
- Join the Rational Developer for Power Hub forum to ask questions and participate in discussions.
- Get connected with your peers and keep up on the latest information in the Rational community.
- Rate or review Rational software. It's quick and easy.
- Share your knowledge and help others who use Rational software by writing a developerWorks article. Find out what makes a good developerWorks article and how to proceed.
- Follow Rational software on Facebook, Twitter (@ibmrational), and YouTube, and add your comments and requests.
- Ask and answer questions and increase your expertise when you get involved in the Rational forums, cafés, and wikis.
Dig deeper into Rational software on developerWorks
Get samples, articles, product docs, and community resources to help build, deploy, and manage your cloud apps.
Experiment with new directions in software development.
Software development in the cloud. Register today to create a project.
Evaluate IBM software and solutions, and transform challenges into opportunities.