In this article I want to cover the development of some Jazz Reporting Service (JRS) custom reports and dashboard widgets to manage and track the progress of the quality metrics and test execution by the system test organization. We can now build these custom reports very quickly with little or no training with the new JRS Report Builder. In separate installments I will take you step by step through using the Report Builder interface to build the following reports:
- Test Case Approval Status
- Test Requirements Coverage
- Test Execution Results by Iteration
- Test Execution Coverage Status
- Defect Find and Verification Status
- Defects Found by Component
- Open Defects by Severity
Within the IBM Collaborative Lifecycle Management (CLM) solution, there are many ways to implement software development projects. As we are developing new versions of the CLM solution, we internally track our project progress and use the CLM solution for our software development environment. This includes using IBM Rational Team Concert (RTC) for change management, planning, build, and source control management. We use IBM Rational Quality Manager (RQM) for quality and test management and IBM DOORS Next Generation (DNG) for describing and managing our product requirements.
As a large development team we are organized in smaller teams around the product boundaries. We have separate RTC project areas to support the development of Jazz Foundation, RTC, RQM, DNG, and Reporting subsystems. In addition there is a centralized System Testing DNG and RQM project areas that are used to test the entire integrated CLM solution. All of these project areas are “friended” so that artifacts can be linked across these project area boundaries.
Test Case Approval Status
If you are managing a testing effort involving the development of new test cases to cover either new or existing functionality, you will want to track the approval status of your test cases. Once they are approved test cases are ready for execution as an auditable regression test phase (often during the final interval of testing for a release).
To look at the status of just those test cases being used for a given test plan (or set of plans), you want to scope the list of test cases by the test plan that they are linked into. Note that you set the relationship to Required so that it properly limits which test cases show up in the list.
Then you limit your test cases by listing only the test plans for which you want to report status by adding a condition to the test plan that is where the test cases are linked. Select the name attribute of the test plan and chose the test plans you want in your report.
Since test case status is not a default column, you need to add Test Case Status as an attribute column.
Relabel the column to test case status to distinguish it from the Test Plan Status and move it up in the column order to just under the test case URL column.
I would like to look at the results as a horizontal bar graph grouped by status values. Up in the Preview bar, I select Graph instead of Table and refresh the view.
Notice that all the test cases are totaled in a single vertical bar and the test case project area name is the label on the x-axis. This is not what I wanted so I change the categories on the x axis to Test Case Status and select “Count values of a single attribute” with a value of Test Case Status. I also set the orientation to Horizontal to get the graph I wanted.
By clicking the Save button, the report builder interface moves me over to the Name and share section where I name the report Test Case Status and fill out a brief description of the report and tag it with my name so I can share it with others. Finally I click the Save button again and this saves my report.
Clicking continue moves me to the Run report step and displays the final report.