Lesson 6: Analyze the test suite execution records

In the previous lesson, you logged in as the tester and ran the test suite execution record. The test suite execution stopped because the first test case did not pass. In this lesson, you will log in as the test manager, Tammy, review the test plan progress and analyze the test execution records.

About this task

You review the progress being made against the Development Test test plan and view the results of the execution after discovering the testers have encountered some problems during their testing.

Procedure

  1. Log in as the test manager (User ID: tammy, Password: tammy).
    If you are prompted with a project login dialog box, select the JKE Banking (Quality Management) project.
  2. In the main menu, click Planning > Browse Test Plans.
  3. In the View As menu, select Execution.
    The Browse Test Plan view changes to display the columns of information that are associated with the Execution view.
  4. Scroll over the status bars to display details of the progress made against the Development Test test plan. Hover over the Progress Report-Points status bar in the Test Suite Points column for the Development Test test plan.
    The test suite execution does not seem to be going well as all the points that have been attempted have either failed or been inconclusive.

    Progress Report - Points status bar details

  5. Click the Details link in the Progress Report-Points status message to view additional information.
    The View Test Suite Execution Records page opens and displays a list of all test suite execution records in the project. Notice that the Last Result column shows a Failed status, click this icon.
  6. Review the Result Overview section of the Test Suite Execution Result page. This section outlines the overall verdict of the test suite and includes summary information about the test suite execution, such as the execution mode (sequential or parallel) and whether the suite execution stopped because of a test case that did not pass.
    In this scenario, you notice that the Was suite execution halted? entry is marked as Yes.
    You know that the suite execution stopped, but you do not know which test case failed or at which step it failed.
  7. In the Test Suite Execution Result page and review the Result Details section. The Result Details section shows the list of all test cases that ran as part of the suite execution and includes a link to the test case result that is associated with the suite test run.
    You can see that the Dividend Allocation by Percentage test case failed and the suite execution stopped before running the Allocate Dividends to a Single Cause test case.
  8. To further analyze why the Dividend Allocation by Percentage test case failed, click the Failed link in the Actual Result column.
    The Execution Result page opens.
  9. Review the Result Details section of this page. The Result Details section outlines the steps that were executed, and the results for each step, so that you can see at which step a test failed.
    From the chart, you can see that the test failed at step 1 and that all other steps in the test were reported as inconclusive.

    results chart

Results

By analyzing your test runs, you can identify trends and spot where bottlenecks exist in your project.

Lesson checkpoint

In this lesson, you learned the following:
  • View the execution progress for a particular test plan while viewing a list of test plans.
  • View the results of a test suite execution record when the test fails.
  • See the test case that failed for a test suite execution record.