Rolling up results for test case execution records

You can create a consolidated test case result by rolling up the results of previous executions of the same test case execution record. The new rolled-up result is formed by merging previous results of the same test case execution record.

About this task

You can combine the previous results of a test case execution record and create a new result.

You can use the roll-up feature for a manual test script, which has test data associated with it. When you add the test script to a test case and run the test case, you can select the test data in each row and execute the script.

For a test script with three rows of test data, the test case is executed three times after you select test data from each row. You can combine the result of the test data by using the roll-up operation and create a consolidated result. If the result of any test data fails, you can rerun the test case and select the test data row that failed in the previous execution and create a test case execution result. You can take the previous and the current results and roll up the result to form a new result.

Consider a test case that has a manual test script associated with it. The test script has five manual test script steps. When you run the test case, two steps are ready and you mark them as passed. The remaining steps are set as deferred. A new test case execution record is created and the overall result of the test case is deferred. Similar to deferred step results, skipped step results are not included in the roll up. You can mark a step as either deferred or skipped if it already passed in a previous execution.
Note: You can enable the preference to execute the steps in any order from the Manage Project Properties page. For more information, see Managing Project Properties. You can also enable a preference to view the previous results of a test case execution record in the execution console.

When you rerun the test case, you can view the previous result of the steps and you can directly run the steps that are deferred. You mark the remaining steps as passed and stop the execution. You can now combine the earlier two results and create a new result by using the Roll Up option, which then becomes the current result of the test case. The rolled-up result contains verdicts from the most recent execution result. If the verdict is missing or deferred for some steps, the previous result is used as the verdict for that step. For more information, see Managing Project Properties.

If the results that participate in the roll-up operation are associated with the same build record, the rolled-up result is also associated with the same build record.

Note:
  • You can use the roll-up feature for manual test scripts. If multiple test scripts are associated with a test case, the test script that is associated with the latest test case result is used for the roll-up operation.
  • The roll-up operation does not work if you add or remove steps in between the current and previous executions of a test case execution record.
  • In progress results are not included in the roll-up operation. You must either complete or cancel the execution to include the result in the roll-up operation. You must enable the When test case executions are canceled or stopped, set unexecuted steps as Deferred execution preference on the Manage Project Properties page before you cancel the execution.

For more information, see this video.

Procedure

  1. In the Test Case Execution Records section of a test case, click the Run icon (Run) to run an existing test case execution record or create a test case execution record.
  2. In the test case execution console, provide the verdict for the steps as passed, failed, or deferred.
  3. Rerun the test case execution record.
  4. In the test case execution console, execute the remaining steps and provide the verdict as passed.
  5. In the Test Case Execution Records section, click the Action Menu icon (Expand/Collapse).
  6. Select Roll Up Test Case Results and then click Roll Up.
  7. Optional: You can also create a rolled-up result by clicking Roll Up with Options. On the Select test case results to roll up page, select the test case results to use for the roll-up operation.
  8. Click OK.

    A new result is created. You can view the result details by clicking the result in the Last Result column. The start time of the result is taken from the first execution and the end time of the result is taken from the latest execution of the test case. You can view the details of the newly created result in the Result Details section of a test case execution record.

  9. Optional: You can click the Change column display settings icon (Change display settings) and add the Last Rolled-Up Result column in the test case execution record table view. You can view the status of the last rolled-up result.