Summary reports

Summary reports provide information about the status of testers and the test effort.

Purpose

Summary reports answer the following questions:

  • What is the status of our testing effort for this test environment?
  • What are the results of each tester’s testing activities?
  • How much testing do we have left to do in this iteration?
  • How many tests have passed (or failed) for this test plan?
  • How much testing does each tester have left to do?
  • What other work does each tester have pending?

The reports included under Summary are essentially execution reports. Execution reports provide information about the status of the test effort for one or more test plans. Only test case execution records (TCERs) that specify one of the test plans that you select in the test plan parameter for the report run are included in the report.

Summary reports reflect information stored in TCERs and the last test case result for each TCER. For example, the points attributed to a TCER are actually contained in the last test case result that is associated with the TCER.

Progress is measured in either the number of TCERs or the number of points in the last test case results of the TCERs. Points are a measure of execution effort specified in the Weight field of a test case and its TCERs. The points in a test case result can be distributed to more than one result status. Consequently, a TCER in which the last result is passed might have both passed and blocked points attributed to it.

In the Execution by Test Schedule (Live) report, you can select optional parameter values to narrow the results included in the report.

Remember: The tester reports (Tester Report using TCER Count and Tester Report using Weight) rely on data pulled from the data warehouse, so make sure that the data collection jobs have been run successfully.
Table 1. Summary report descriptions
Report Name Description
Execution by Test Schedule (Live)

For one or more selected test plans, this report presents a breakdown of points for each TCER that matches the selected parameters. The results are presented in a table that is broken down by test plan, test iteration, tester, test environment, and test case, and shows the number of test runs attempted, the total weight, the points attempted or not started, and the point distribution by test result status (passed, failed, blocked, and so on).

Note: The weight column shows the weight of the last test case result unless the TCER has not been run, in which case the weight column shows the weight of the TCER.

The required Group Order parameter determines the order of the breakdown by iteration, tester, and test environment. The default group order is Iteration, Tester, Environment.

Only test cases that have at least one TCER (even if not run) are included in this report.

This report answers these questions:
  • “What is the status of our testing effort for this test environment?”
  • “What are the results of each tester’s testing activities?”
  • “How much testing do we have left to do in this iteration?”
  • “How many tests have passed (or failed) for this test plan?”
You can narrow the results included in the report by selecting values for these optional parameters:
  • Test Milestone
  • Test Case
  • Machine
  • TCER Owner
  • TCER State
  • Test Plan Category Type
  • Test Plan Category Name
  • Test Case Category Type
  • Test Case Category Name
Tip: For each test plan:
  1. Depending on the time frame, many points not started might indicate that you are falling behind in the test effort overall or within a specific iteration.
  2. Many failed or blocked points might point to an area that warrants more focused testing.
  3. Few TCER attempts within a specific test environment might indicate that there is a problem with the stability or availability of the test environment.
  4. If there is an imbalance of TCER attempts or points attempted among testers, consider reassigning some testing activities to other testers.
Tester Report using TCER Count

For one or more selected test plans and TCER owners, this report displays a bar graph and table of test result status and a list of unfinished work items.

The bar graph shows, for each TCER owner, bars representing the number of TCERs for each test result status (passed, failed, blocked, and so on). The bar graph is followed by a table representation of the same information.

Move the cursor over a segment of a bar graph to see how many TCERs are included in that segment. Click a segment of the bar graph to see a list of the TCERs that are represented by that segment.

The list of unfinished work items includes all unresolved work items that are owned by a selected TCER owner (whether or not the work items are related to a selected test plan).

This report answers these questions:
  • “What are the results of each tester’s testing activities?”
  • “How much testing does each tester have left to do?”
  • “What other work does each tester have pending?”

This report pulls data from the data warehouse. Make sure that the data collection jobs have been run successfully.

Tip: For each tester:
  1. Depending on the time frame, many TCERs not started might indicate that a tester is blocked in some way (by other work priorities, technical difficulties, or an unavailable or unstable test environment, for example).
  2. Many failed or blocked TCERs might point to an area that warrants more focused testing.
  3. If a tester has many unfinished (unresolved) work items, consider reassigning some work items or testing activities to other testers.
Tester Report using Weight

For one or more selected test plans and TCER owners, this report displays a bar graph and table of test result status and a list of unfinished work items.

The bar graph shows, for each TCER owner, a stacked bar representing the TCER points that are attempted and not started, followed by bars representing the number of TCER points for each test result status (passed, failed, blocked, and so on). The bar graph is followed by a table representation of the same information.

Move the cursor over a segment of a bar graph to see how many TCERs are included in that segment. Click a segment of the bar graph to see a list of the TCERs that are represented by that segment.

The list of unfinished work items includes all unresolved work items that are owned by a selected TCER owner (whether or not the work items are related to a selected test plan).

This report answers these questions:
  • What are the results of each tester's testing activities?
  • How much testing does each tester have left to do?
  • What other work does each tester have pending?

This report pulls data from the data warehouse. Make sure that the data collection jobs have been run successfully.

Note: For each tester:
  1. Depending on the time frame, many points not started might indicate that a tester is blocked in some way (by other work priorities, technical difficulties, or an unavailable or unstable test environment, for example).
  2. Many failed or blocked points might point to an area that warrants more focused testing.
  3. If a tester has many unfinished (unresolved) work items, consider reassigning some work items or testing activities to other testers.