Running a suite concurrently

You can run a suite to run multiple tests and other suites in parallel to expedite the testing process.

You must have completed the following tasks:
  • Ensured that you are assigned a Team Space Owner, Project Owner, or Member with a Tester role to create or edit a test resource.
  • Ensured that a suite is available in your project.
  • Added tests or other suites to the suite.

You can run a suite concurrently to run multiple suites and tests simultaneously across several pods. You can set up the maximum number of concurrent pods, which defines how many suites or test assets can run in parallel. In the default set up, suites run their child assets sequentially on a single pod, with only one test asset running at a time.

When you select the Run in Parallel option, the Continue on Fail checkbox is disabled. Whether a test passes or fails, all test assets run independently and concurrently across the available pods.
Restriction: To run server suite assets on the Safari browser, you must use DevOps Test Runtime agents. When you use Safari, the tests in the suite run sequentially even when you selected the Run in Parallel option.

As the test runs, the three color-coded boxes display the counts of passed steps, failed steps, and errors. You can view the step details of the first failed step or error step by clicking the respective color-coded boxes. Also, you can view checkmarks for the steps that pass and crossmarks for the steps that fail.

  1. Log in to Test.
    The Projects page of the initial team space is displayed.
  2. Click My projects > project_name to open the project that contains the test assets.
    The Overview page of the project is displayed.
  3. Click Author > Test Editor.
    The Test Editor page and the test navigator panel are displayed.
  4. Click the .dts.yaml file in the list of Suites to open the suite.
    The tests or suites are displayed on the Suite Editor page.
  5. Click the Configure settings and data step.
    The Settings pane is displayed.
  6. Select the Run in parallel checkbox to enable multiple tests to run at the same time.
    The Continue on Fail option gets disabled.
  7. Enter the value in the Maximum Concurrent Pods field.
    Note:

    The Maximum Concurrent Pods setting defines the number of test assets from a suite that can run in parallel and the minimum value must be two. A lower value reduces server resource usage but increases run time, and a higher value speeds up the run time at the cost of higher resource consumption.

  8. Click the Run all icon icon, and then select the Run all > Run all on Default Cluster option.

    You can also select the Run all on Personal Agent option if the personal agent is in a ready state.

    You can view the following stages on the progress bar during the test run:
    • In transition: The test setup is in progress.
    • Running: The test is actively running.
    • Complete: The test run is completed. This section also displays the time taken for test run.
    • During the Running phase, you can view the following details:
      • A checkmark check mark icon after each test or suite that passes, and a crossmark cross mark icon for any failed test or suite.
      • The Results tab displays the test as In Progress during the test run.
    • After the test run is completed, you can view the following results:
      • The Results tab displays the test as Pass or Fail.
  9. Click Analyze to open the Results page, where you can view the results, reports, and logs of the completed test run, and perform the following steps:
    1. Verify that a secondary label appears for the secondary executions.
    2. Click on secondary and main/primary execution.
    3. Verify that the Test Log and Functional Report options are displayed only for the main/primary execution and does not display for the secondary executions.
You have run a suite concurrently.

You can view the following information:

  • The details of the suites and individual test results from the Run Details tab.
  • The console output, which includes error messages and logs. See Accessing console output.
  • The test results, reports, and logs from the Results page after the test completes the run.
  • The summary of the test run, which includes passed or failed status, count of passed or failed tests or test steps, and the verification points in the functional report. See Viewing a Functional Report.