Run All Published Tests

Description

The Run All Tests endpoint executes all of the published tests within an API Hook's scope.

HTTP Request


    POST <API-Hook-URL>/tests/run

Sample Method Invocation


    curl -X POST \
    -H X-API-Key:{API-Key} \
    -H X-API-Secret:{API-Secret} \
    -H Content-Type:application/json \
    -d '
        {
          "options": {
              "allAssertions": boolean,
              "JUnitFormat": boolean
          },
          "variables": {
              string: string,
          }
        }
      ' \
    <API-Hook-URL>/tests/run

Parameters

Header Parameters


API-Key: A valid API Key for the Test Organization this API Hook belongs to.
API-Secret: A valid API Secret for the Test Organization this API Hook belongs to.

Request Body

MIME type: application/json



    {
    options: {
        allAssertions: boolean,
        JUnitFormat: boolean
    },
    variables: {
        string: string,
    }
    }

Body Parameters

  • options (Only required if providing a child element)
    • A wrapper object for the containing options on how what data should be returned in the response, or how the response should be formatted.
    • Child Elements:
      • allAssertions (Optional)
        • When false, only the results of failing assertions in the test are returned. When true, all assertions in the test are returned. allAssertions is false by default.
      • JUnitFormat (Optional)
        • When false, application/json format is returned for the test results. When true, text/xml JUnit format is used for the test results (see text/xml response). JUnitFormat is false by default.
  • variables (Optional)
    • The variables section is used to pass a list of named variables and their values into the tests being run. It is only used by tests that have been written to make use of variables, and have values stored alongside them. Declaring the variable name as the object key, and its new value as the key's value (as type string), will overwrite the current value of the variable in the test which shares its name with the object key.

Response Content

MIME type: application/json


application/json is the default response from the endpoint.

    [
    {
      testRunId: string,
      testId: string,
      testName: string,
      location: string,
      initiated: string,
      duration: integer,
      status: enum['passed' or 'failed'],
      reportUrl: string,
      results: {
        warningsCount: integer,
        failureCount: integer,
        httpFailures: array[object],
        criticalFailures: array[object],
        assertions: array[object]
      }
    }
    ]

Response Description

  • testRunID
    • ID of the test run.
  • testId
    • ID of the test which was executed.
  • testName
    • Name of the test.
  • location
    • Location of the agent which excuted the test.
  • initiated
    • Date and time when the test was initiated.
  • duration
    • Total duration of the test run in seconds.
  • status
    • Overall result of the test.
  • reportUrl
    • URL to the test run report.
  • results
    • A wrapper object containing the results of the test run.
    • Child Elements:
      • warningsCount
        • Count of the HTTP warnings returned by the test assertions.
      • failureCount
        • Count of the HTTP status code responses that are greater than 400.
      • httpFailures
        • Description of failures whose HTTP Status response code is greater than or equal to 400.
      • criticalFailures
        • Description of failures whose HTTP Status response code is greater than or equal to 500.
      • assertions
        • A description of the assertions executed in the test. By default, only failing assertions will be listed.

MIME type: text/xml


To receieve a text/xml response, the optional JUnitFormat variable in the request body must be set to true. JUnit responses can be useful when you wish to incorporate your test reults with a build pipline such as Jenkins, via the JUnit plugin. 

    <?xml version="1.0" encoding="UTF-8"?>
    <testSuite name="" tests="" disabled="" errors="" failures="" timestamp="" time="">
    <testcase name="" classname="" time="">
      <failure message="" type="" />
    </testcase>
    </testSuite>

Response Description


  • <testSuite>
    • Child Elements: <testCase>
    • Attributes:
      • name
        • Test Suite name.
      • tests
        • Number of published tests ran.
      • disabled
        • Number of disabled tests.
      • errors
        • Number of errors.
      • failures
        • Number of tests within the test suite that failed.
      • timestamp
        • Date and time the tests were ran.
      • time
        • Total duration of all test runs in seconds.
  • <testCase>
    • Child Elements: <failure>
    • Attributes:
      • name
        • Test name.
      • classname
        • ID of the test which was executed.
      • time
        • Total duration of the test run in seconds.
  • <failure>
    • Attributes:
      • message
        • Failure message containing the expected and actual response.
      • type
        • Type of failure.

Status Codes

 200: Success
400: Missing Authorization Headers
401: Invalid Authorization Headers