Test suite execution workflow

A test suite is a collection of test cases that are grouped for test execution purposes. Test suites can include both manual and automated tests, but they are most beneficial when you have a series of automated tests that you can run without supervision. When running a suite, you can specify whether to run the test cases in the suite sequentially or in parallel. The following scenario presents one possible workflow for test suite execution.

Figure 1. Workflow diagram for test suite execution
Click here for information about creating test cases Click here for information about developing test scripts Click here for information about adding test scripts to a test case Click here for information about adding an existing test case to a test plan Click here for information about creating test suites Click here for information about adding an existing test suite to a test plan Click here for information about adding an existing test cases to a test suite Click here for information about creating test suite execution records Click here for information about running test suite execution records Click here for information about analyzing execution results
  1. Create test cases.
  2. Optional: Create test scripts, and associate them with the test cases.
    Note:

    Test cases are required for test suite execution, but each test case does not need to have an associated test script.

  3. Optional: Add the test cases to a test plan.

    A test plan helps to organize test execution activities, but it is not a requirement for test execution. By including both the test cases and the test suite in a plan, you can plan for the execution of the suite as part of the testing activities for the plan. Test suite execution records run in the context of the test plan and test plan iterations. Also, all test suite execution results (including test case execution results) are associated with the test plan. All reports that you run against these results will include the associated test plan information.

  4. Analyze the existing test cases and test scripts in your environment to determine which ones will be part of the new test suite.
    Ideally, each test case in a suite represents a piece of a scenario, such as the elements that simulate the user completing a transaction. For example, a test suite could contain four test cases, each with a separate associated test script:
    • Test case 1: Log in
    • Test case 2: Select an organization
    • Test case 3: Allocate dividends
    • Test case 4: Log out
  5. Create a new test suite.
  6. Optional: Add the test suite to a test plan.

    As mentioned in step 3, a test plan is not a requirement for test suite execution, but it can help you organize testing activities. If you add the test suite to a test plan, ensure that the test cases that are included in the test suite are also part of the same plan.

  7. Add existing test cases to the test suite.

    Note that the order of the test cases in the Test Case section of a test suite determines the order in which they are executed in a sequential test suite execution.

    When you add test cases to a test suite, you can optionally assign owners to each test case in the suite. When you run a test suite that has owners assigned to test cases, the test suite execution page shows the owner for each test case in the suite. Testers can filter on the owner field to see the test cases in the test suite that they should execute.

  8. Create a single test suite execution record for the test suite or generate multiple test suite execution records for the test suite automatically.
  9. Optional: Specify an environment that is specific to the test plan for the suite to be executed. If you only want the suite to test one environment, specify the test environment at the test suite execution record level. If you want to test a mixed environment, you must specify the test environments in the individual test cases and not in the test suite execution record. Essentially, the environment on the test suite execution record is the default specification unless it is overriden at the test case level.
    Note: You cannot change the associated Test Plan.
  10. Run the test suite. You can run a test suite in the following ways:
    • From the Test Suite Executions Records section of a test suite, select an existing test suite execution record, and click the Run icon.
    • Open a test suite, and click the Run icon. Select an existing test suite execution record, or specify that a new one be created.
  11. In the Run Test Suite dialog box, specify the execution settings:
    • Run this suite in parallel: When you run your test suite, you can run all test cases at the same time (parallel mode), or run them one after another (sequential mode). In this workflow, our test cases represent a sequence of activities (you must log in before you can select an organization, for example). To run this test correctly, clear the Run this suite in parallel check box to select sequential mode. Sometimes you might want to run all of your test cases at the same time. For example, if the scripts do not depend on each other and you do not need to execute them in any particular order. In that case, you can choose parallel mode.
    • Stop suite execution if any test does not pass: With sequential test suite execution, you can stop the test suite if one of the test cases does not pass. For example, if the test to log in to the application fails or is blocked, the test suite execution stops before continuing with the test to select an organization.
    • Create Result without Execution: Use this option to record results for the test suite without executing the test scripts.
    Optionally, you can choose test cases to exclude from the test suite execution:
    • If you selected an existing test suite execution record, you can choose to skip the execution of any test cases that passed during the previous execution of the test suite execution record by clicking the Exclude Passed Test Cases (Exclude Passed Test Cases) icon.

    • To skip specific test cases, select the test cases in the table and click the Exclude from Execution (Exclude Execution) icon.

  12. View the test suite execution results, and submit defects as needed.
  13. Update the test suite execution results as needed and save the results.