Running test suites
A test suite is a collection of test cases that are grouped to be run together. You can run a test suite by opening the test suite or by opening the Browse Test Suites window.
About this task
After testing starts, you can pause test suite runs from the Test suite execution console. You might pause test suite runs when the system that you are testing needs minor adjustments before testing can proceed.
When you pause a test suite run, any manual test runs that are part of the suite are also paused. If you pause a test while an automated test is running, that test is completed; however, automated tests that are not started do not run. In either case, the test suite is paused. You can resume paused test suites from the Test suite execution console, Test suite execution record list view, or Test suite execution record editor.
You can stop test suites from the Test suite execution console. Unfinished test cases in that suite are marked incomplete. Stopped test suites cannot be resumed.
- Open a test suite. Note: You cannot run a test suite until it is saved.
- In the upper right of the test suite window, click the Run Test Suite icon () or use the keyboard shortcut Ctrl + Shift + X.
In the Test Suite Execution Record section, either select a test suite
execution record or create an execution record by accepting the default option, New Test
Suite Execution Record.
If you create an execution record, select the test plan, test iteration, and the test environment.Note: If you already have an execution record that uses the same test plan, test iteration, and test environment that you specified, that execution record is used. A new execution record is not created.
If you run a test suite that has the Run a test script option enabled,
the automation script that is associated with the test suite is run. In the Run Test
Script dialog box, you can select the following options:
When you run the automation script, the result of the test cases that are associated with the test suite are displayed in the Result Details section of the test suite result.
- Schedule execution
- Create a result without executing
- Modify execution variable values
Select a channel.
During the run of the test suite, the default script for the channel is chosen by using the keywords in the script. The available channels are based on the selected test environment.
Select a test cell. If the test suite includes automated test cases, you can specify the test
cell to run the test suite.
All the automated test cases are distributed among the test automation adapters that are in the specified test cell.
If the test suite contains test cases that have automated test scripts, you can select multiple
test cases and assign the same adapter to all of them. Select the check boxes that are next to the
test cases, and then, in the Machine column, click the Change
Machine icon ().
Note: Most automated test adapters can run only one script at a time. If you select the same adapter for multiple test cases, the scripts on that adapter are run in sequence. To run scripts in parallel, you must select a different adapter for each script.
Specify which test cases to exclude from the test suite run. If you do not exclude test cases,
all the test cases in the suite are included. To skip test cases, complete one or both of the
All the test cases that you selected are marked with an icon in the Exclude from Execution column and are not included in the test suite run. If you decide to include all the test cases in the test suite run, select the test cases again and click the Exclude from Execution icon ().
- If you selected a test suite execution record, you can skip any test cases that passed when the test suite execution record last ran. Click the Exclude Passed Test Cases icon ().
- To skip specific test cases, select the test cases in the table and click the Exclude from Execution icon ().
- Optional: Specify a default test script for each test case that runs.
- To associate the results of this test suite run with a build record, next to the
Build Record option, click the Change Associated Build
icon () and select a build record. The execution result from this test run is automatically linked to the build record that you select.
- To schedule the test suite to run at a future time, select Schedule Execution and select a date and time.
- Select whether to run the test cases that are in the suite in parallel.
- If you selected the option to run the tests in a sequence,
you might select more options:
- If your test scripts use execution variables and you want to pass the values between scripts during the test suite run, select Pass execution variables between scripts.
- To specify that the test suite must be stopped if a single test case does not pass, select Stop suite execution if any test does not pass. This option is useful if you are running a test case in a test suite that depends on the success of previous test cases.
- To record a result for this test without running the scripts, select Create Result without Execution and then select a result.
- To temporarily modify the values of the execution variables that are defined in your test suite, click Modify Execution Variable Value(s).
- Open the Test suite execution console
by clicking Run. If you are running tests in a sequence, the first test case in the sequence is shown in the console. If you run tests in parallel, the automated test cases start to run and the manual test cases or test cases without scripts are shown in the console.
If the first test case in the suite is a manual test, click Start Manual
Test. If the first test case in the suite does not have an associated test script, start
the test by clicking Run Test Case in the Test Case
If the first test case in the suite is an automated test, the test runs automatically.
For manual tests, follow the steps in the manual test execution window
and, when you are finished, click Close. If a test case does not have a
script, record a result for the test case and then click Save and
If you have manual scripts or test cases without scripts and you are running the test in parallel, you can return to the Test suite execution console at any time. You can start to run another test case in the suite.
- When a test case is completed, you can view the execution result for the test case by clicking the icon in the Result column. The execution result includes the editable Actual Result field and more details about the testing artifacts and testing environment that is associated with the test case.
Update the execution results as needed, and be sure to save your updates.
For example, you can provide a detailed description of the test results in the Result Details section.Note: The Results Details section of the test suite result displays a snapshot of the test case results when the associated test suite was executed. If any of the test case results are deleted, the entries remain in the Result Details section to indicate that the test cases were executed, even if there are no details on the deleted test case results.
- To submit a defect, in the Change Requests section, click the Create New Defect icon () or Link to Existing Defect icon (). In the defect form that opens, type information about the problem, and then save the page.
- Run the remaining test cases in the suite. To finish running the test suite later, or if another tester is responsible for the next test case, close the page. You can return to the test suite execution later.Note: If you close the test suite execution console during a test suite run, you can return to it in any of these ways:
- Click Last Result icon. and then click the
- Click Test Suite Progress column, click the link. and find the test case that must be run as part of the test suite. In the
- Open a test plan and in the Test Suite Execution Records section, click the Last Result icon.