Execution reports

Execution reports provide information about the status of the test effort, including overall software quality and test execution progress against the planned progress.

Purpose

Execution reports answer the following questions:

How much work is complete, how much work is remaining, and is progress being made as expected?
Which test plans have high numbers of failed points or test case execution records (TCERs)?
Which test plans or TCERs have high numbers of defects?
Which milestones of a test plan have high numbers of failed points or test execution records?
Which machines have high numbers of failed points or test execution records?
Which testers have recorded high numbers of failed points or test execution records?
Which test case categories have high numbers of failed points or TCERs?
Which test suites have high numbers of failed points or test suite execution records (TSERs)?
Which TCERs or TSERs have high numbers of failures?

In many of these reports, you can select optional parameter values to narrow the results included in the report.

Note: While many of these reports access live data, some reports rely on data pulled from the data warehouse. Make sure that the data collection jobs have run successfully before running the following reports:

Execution Trend
Execution and defects by owner

Execution report descriptions

Report Name	Description
Execution Status by Machine using TCER Count (Live)	Use this report to display the execution status by test machine, measured against the test case execution record count. Note: This report is removed in version 5.0 and later. Project areas that migrate from a version previous to 5.0 still include the report, but new project areas do not. It answers the question, Which machines have high numbers of TCER failures? Failures on a particular machine might indicate instability in a certain area of the product under test. Each bar in the graph shows the number of test case execution records associated with a test machine, as well as their execution statuses. If the test case execution records are in varying states, the bar will be stacked with those status categories. Move the cursor over a category to see the number of test case execution records in that grouping. Tip: You can click any bar in the chart to display a list of the corresponding TCERs for further investigation or diagnosis.
Execution Status by Machine using Weight (Live)	Use this report to display the execution status by test machine, measured against weight. Note: This report is removed in version 5.0 and later. Project areas that migrate from a version previous to 5.0 still include the report, but new project areas do not. It answers the question, Which machines have high numbers of failed points? Many failures on a particular machine might indicate instability in a certain area of the product under test. The report uses the weight distribution defined in the execution result. Therefore, there might be points allocated to a different result than the overall result of the TCER execution. For example, the overall result of an execution could be fail, while some points are marked as passed. Each bar in the graph shows the weight, or effort, required to complete all of the test case execution records associated with a test machine. If the test case execution records are in varying points categories, the bar will be stacked with those status categories. Move the cursor over a category to see the cumulative weight for that grouping. Tip: You can click any bar in the chart to display a list of the corresponding TCERs for further investigation or diagnosis.
Execution Status by Milestones using TCER Count (Live)	Use this report to display the execution status by test milestones of a test plan, measured against the test case execution record count. It answers the question, Which milestones of a test plan have high numbers of TCER failures? Failures on a particular milestone might indicate instability in a certain area of the product under test. Each bar in the graph shows the number of test case execution records associated with a milestone in the selected test plan, as well as their execution statuses. If the test case execution records are in varying states, the bar will be stacked with those status categories. Move the cursor over a category to see the number of test case execution records in that grouping. Tip: You can click any bar in the chart to display a list of the corresponding TCERs for further investigation or diagnosis.
Execution Status by Milestones using Weight (Live)	Use this report to display the execution status by test milestones of a test plan, measured against weight. It answers the question, Which milestones of a test plan have high numbers of failed points? Many failures on a particular milestone might indicate instability in a certain area of the product under test. The report uses the weight distribution defined in the execution result. Therefore, there might be points allocated to a different result than the overall result of the TCER execution. For example, the overall result of an execution could be fail, while some points are marked as passed. Each bar in the graph shows the weight, or effort, required to complete all of the test case execution records associated with a milestone in the selected test plan. If the test case execution records are in varying points categories, the bar will be stacked with those status categories. Move the cursor over a category to see the cumulative weight for that grouping. Tip: You can click any bar in the chart to display a list of the corresponding TCERs for further investigation or diagnosis.
Execution Status by Owner using TCER Count (Live)	Use this report to display the execution status by owner, measured against the test case execution record count. It answers the question, Which testers have recorded high numbers of failed TCERs? Many failures found by a particular tester may indicate instability in a certain area of the product under test. Each bar in the graph shows the number of test case execution records associated with an owner, as well as their execution statuses. If the test case execution records are in varying states, the bar will be stacked with those status categories. Move the cursor over a category to see the number of test case execution records in that grouping. Tip: You can click any bar in the chart to display a list of the corresponding TCERs for further investigation or diagnosis.
Execution Status by Owner using Weight (Live)	Use this report to display the execution status by owner, measured against weight. It answers the question, Which testers have recorded high numbers of failed points? Many failures found by a particular tester may indicate instability in a certain area of the product under test. The report uses the weight distribution defined in the execution result. Therefore there may be points allocated to a different result than the overall result of the TCER execution. For example, the overall result of an execution could be fail while some points are marked as passed. Each bar in the graph shows the weight, or effort, required to complete all of the test case execution records associated with an owner. If the test case execution records are in varying points categories, the bar will be stacked with those status categories. Move the cursor over a category to see the cumulative weight for that grouping. Tip: You can click any bar in the chart to display a list of the corresponding TCERs for further investigation or diagnosis.
Execution Status by Test Case Category using TCER Count (Live)	Use this report to display the execution status by test case category, measured against the TCER count. It answers the question, Which test case categories have high numbers of failed TCERs? Depending on the categories that you have defined, many failures found for a particular category might indicate instability in a certain area of the product under test. Tip: You can click any bar in the chart to display a list of the corresponding TCERs for further investigation or diagnosis.
Execution Status by Test Case Category using Weight (Live)	Use this report to display the execution status by test case category, measured against weight. It answers the question, Which test case categories have high numbers of failed points? Depending on the categories you have defined, many failed points found for a particular category may indicate instability in a certain area of the product under test. The report uses the weight distribution defined in the execution result. Therefore there may be points allocated to a different result than the overall result of the TCER execution. For example, the overall result of an execution could be fail while some points are marked as passed. Tip: You can click any bar in the chart to display a list of the corresponding TCERs for further investigation or diagnosis.
Execution Status by Test Suite using TSER Count (Live)	Use this report to display the execution status by test suites, measured against the TSER count. It answers the question, Which test suites have high numbers of failed TSERs? Many failures against a particular test suite might indicate instability in a certain area of the product under test. The bar chart shows the TSER count of the last execution results of all test suite execution records of given test suites, including the points not started. By clicking a bar in the report, you can drill down to the TSER Listing (Live) report to show the test case result details for execution results of the test suite's test suite execution records.
Execution Status by Test Suite using Weight (Live)	Use this report to display the execution status by test suites, measured against weight. It answers the question, Which test suites have high numbers of failed points? High numbers of blocked points might indicate that problems are preventing testing progress. The bar chart shows the weight distribution of the last execution results of all test suite execution records of given test suites, including the points not started. If one test suite execution record is never run, its weight number is counted into points not started. By clicking a bar in the report, you can drill down to the TSER Listing report to show the weight distribution of execution results of the test suite's test suite execution records where the point matching to the point category of the bar is not 0.
Execution Status using TCER Count (Live)	Use this report to display the execution status by test plan, measured against the test case execution record count. It answers the question, Which test plans have high numbers of failed TCERs? Many failures against a particular test plan might indicate instability in a certain area of the product under test. Each bar in the graph shows the number of test case execution records associated with a test plan, as well as their execution statuses. If the test case execution records are in varying states, the bar will be stacked with those status categories. Move the cursor over a category to see the number of test case execution records in that grouping. You can view a list of the test case execution records in a particular state by clicking that segment of the bar. Viewing pending work
Execution Status using Weight (Live)	Use this report to display the execution status by test plan, measured against weight. The report uses the weight distribution defined in the execution result. Therefore, there might be points allocated to a different result than the overall result of the TCER execution. For example, the overall result of an execution could be Fail while some points are marked as passed. It answers the question, Which test plans have high numbers of failed points? Many failed points against a particular test plan might indicate instability in a certain area of the product under test. Each bar in the graph shows the weight, or effort, required to complete all of the test case execution records associated with a test plan, as well as their execution statuses. If the test case execution records are in varying states, the bar will be stacked with those status categories. Move the cursor over a category to see the cumulative weight for that grouping. Tip: You can click any bar in the chart to display a list of the corresponding TCERs for further investigation or diagnosis.
Execution Trend	Use this report to estimate the actual test execution progress against the planned progress. The report uses weight to determine how much work is complete, how much work is remaining, and whether progress is being made as expected. It answers the questions, How much work is complete, how much work is remaining, and is progress being made as expected? This report displays a quick indication of where you are in your actual testing versus your plan. To generate the report, you must have both the planned and actual start and end dates defined. If you do not know the actual dates yet, use the same dates that you defined for the planned start and end dates. You can later update the fields with the actual dates. This report pulls data from the data warehouse. Make sure that the data collection jobs have been run successfully. Tip: Make sure that there is valid data in the test schedules of the test plan. Also make sure that the TCERs are associated with the test plan and test milestones in the report. The days behind or ahead of schedule information is based on the date of the last data collection job. Checking project status with the Execution Trend Report
Execution and defects by owner	Use this report to display the status of test case execution records and their associated defects for each owner. It answers the question, Which test plans or TCERs have high numbers of defects? Test Plans or TCERs with many associated work items as this may indicate a comparatively unstable area. This report pulls data from the data warehouse. Make sure that the data collection jobs have been run successfully.
TCER Listing (Live)	Use this report to list test case execution records. From the results, you can link to the test case, test environment, and last execution result associated with a specific test case execution record. Look for TCERs that have high numbers of failures. Also, high numbers of TCERs without owners might indicate that testing work still needs to be distributed across your team.
TCER Status Counts (Live)	Use this report to list the status of execution by points. The results display an overview for points attempted, passed, failed, blocked, and inconclusive based on the test plan. You can open test plans associated with the point categories by clicking the links. The Iteration parameter will be selected based on item id. It answers the question, Which test plans have high numbers of failed points? High numbers of blocked points may indicate that problems are preventing testing progress.
TCER Status Counts Name (Live)	Use this report to list the status of execution by points. The results display an overview for points attempted, passed, failed, blocked, and inconclusive based on the test plan. You can open test plans associated with the point categories by clicking the links. The Iteration parameter will be selected based on name. It answers the question, Which test plans have high numbers of failed points? High numbers of blocked points may indicate that problems are preventing testing progress. Tip: This report may not be displayed by default. You can add this report to the list of deployed report resources in a project or team area, see Deploying new BIRT or EDG report resources.
TSER Listing (Live)	Use this report to display test suite execution records, based on selected parameters. It answers the question, Which TSERs have high numbers of failures? High numbers of blocked TSERs may indicate that problems are preventing testing progress. Also, high numbers of TSERs without owners may indicate that testing work still needs to be distributed across your team. Only the test suite execution records that are included in a test plan in the project area are listed in the report. In addition to the basic information about the test suite execution records, you can see the total number of test cases and how many test cases are passed, failed, blocked, in progress, inconclusive, and not started in the last execution result of the test suite execution record. You can also get the aggregate counts grouped by test plan and test suite in the test plan from the report. The last row of the report summarizes the test cases counts for all of the test suite execution records listed in the report.
TSER Status by Owner using TSER Count (Live)	Use this report to display the execution status of test suite execution records by its owner, measured against the TSER count. It answers the question, Which testers have high numbers of failed TSERs? High numbers of blocked or not started TSERs might indicate that problems are preventing testing progress. The bar chart shows the TSER count for the chosen TSER owner. If the TSER has no owner assigned, this TSER result is counted as Unassigned. By clicking a bar in the report, you can drill down to the TSER Listing report to show the status of the test case results.
TSER Status by Owner using Weight (Live)	Use this report to display the execution status of test suites, showing the points distribution across owners, based on selected parameters. It answers the question, Which testers have high numbers of failed points? High numbers of blocked or not started points might indicate that problems are preventing testing progress. This report focuses on the weight distribution of the last execution results of given test suite execution records, including the points not started. If the test suite execution record is never run, its weight number is counted into points not started. Clicking a bar in the chart opens a TSER Listing report that shows the details for the bar.
TSER Status using Test Case Count (Live)	Use this report to display the execution status of test suites, measured against test case count for each execution result status. It answers the question, Which TSERs have test cases with high numbers of failed TCERs? High numbers of blocked or not started test cases might indicate that problems are preventing testing progress. Only the test suite execution records that are included in a test plan in the project area are displayed in this report. You can see the total number of test cases and how many test cases are passed, failed, blocked, in progress, inconclusive, and not started in the last execution result of the test suite execution record. You can also group the aggregate counts by test plan and test suite in the test plan. The last row summarizes the test case counts for all of the test suite execution records listed in the report.
TSER Status using Weight (Live)	Use this report to display the execution status of test suites, showing the points distribution, based on selected parameters. It answers the question, Which TSERs have high numbers of failed points? High numbers of blocked or not started points might indicate that problems are preventing testing progress. This report focuses on the weight distribution of the last execution results of given test suite execution records, including the points not started. If the test suite execution record is never run, its weight number is counted into points not started.
Test Case Results Details of Test Suite Results (Live)	Use this report to list the test case execution records of the given test suites, based on selected parameters. It answers the question, Which TCERs in TSERs with high numbers of failed points should I analyze? High numbers of blocked or not started points might indicate that problems are preventing testing progress. This report focuses on the test case execution results details of the selected test suite execution result. Only the last execution result of the selected test suites are shown, and the test case execution results according to these test suite execution results are listed. You can also see the total number of points of passed, failed, blocked, in progress, inconclusive, not started, perm failed, and deferred in the last execution result of the test suite execution record.