Contribute in GitHub:

Configuring test cases for Voice Agent Tester

You can create a test case to run automated voice tests against a voice agent by creating a JSON object and then posting that object. You can also do more actions, such as modifying or deleting test cases.

A test case contains the outline of a conversation and is the basic resource that is needed for running tests. The JSON object for a single test case includes metadata about the test along with a JSON array called turns, which contains an array of turn objects. Each turn object contains the expected transcribed audio from the voice agent under test.

Additionally, each turn object contains a response that is synthesized into audio that is sent back to the voice agent under test. The data model also supports the ability to look for substrings and run regex expressions against the transcribed audio.

Creating a test case

To create a test case, first create a JSON object and then do a POST request for the JSON object.

You can use the following example to create your JSON object:

{
    "name" : "full payment flow",
    "description" : "This test verifies the full payment dialog flow",
    "namespace":"default",
    "turns" : [
        {
            "receive":{
                "type": "string",
                "value": "Hi, this is your virtual assistant. How can I help you?"
            },
            "send":{
                "type": "string",
                "value": "I want to make a payment"
            }
        },
        {
            "receive":{
                "type": "string",
                "value": "I can help with that. What is your PIN number?"
            },
            "send":{
                "type": "string",
                "value": "1234"
            }
        }
    ]
}

The following table describes the properties of the JSON object:

Table 1. Properties for the JSON object
Property	Description
name	Specifies the name of the test case
description	Optionally specifies a more detailed description for the test case
namespace	Optionally defines a group to which a test case is bound. If no namespace is specified, the namespace is default.
turns	Specifies an ordered array representing a conversation. Each index is a turn in the conversation represented by a JSON object composed of what you expect to receive and what you would like to send.

The following table describes the supported types for the receive JSON object:

If any of the receive objects don't match the response from the voice agent under test, the test case is marked in the final results as a failure.

Table 2. Supported types for the receive JSON object
Type	Definition
string	Specifies the exact case insensitive string expected from the transcribed audio. This audio is received from the voice agent under test.
substring	Specifies a substring that exists within the transcribed audio. This audio is received from the voice agent that is under test.
regex	Specifies a regex expression that is run against the transcribed audio. This audio is received from the voice agent under test.
context	Specifies a JSON object that contains the property and value to look for in the returned context.
orList	Specifies an array of receive objects, any of which can be matched with the input from the voice agent under test.
andList	Specifies an array of receive objects, all of which must be matched with the input from the voice agent under test.

The following table describes the supported types for the send JSON object:

Table 3. Supported types for the send JSON object
Type	Definition
string	Specifies the text to synthesize into audio and stream back to the voice agent under test. For more information, see the action sequences.
commands	Specifies a Voice Gateway command sequence to initiate back to the voice agent under test. For more information, see the action sequences

Next, create the test case by entering a POST request for the JSON object.

Use the following example:

POST /voice-agent-tester/v1/case

If the POST request is successful, a 201 CREATED response is returned. This response includes the GUID of the test case that you created. You use the GUID to manage the test case.

Additional requests to manage test cases

To get the test case JSON object associated with a specific ID, use a request similar to the following example:

GET /voice-agent-tester/v1/case/<returned GUID of the test case>

The following example shows the JSON object that can be returned on a GET request:

{
  "id":"guid",
  "rev":"version",
  "name" : "full payment flow",
  "description" : "This test verifies the full payment dialog flow",
  "namespace":"default",
  "turns" : [
      {
          "receive":{
              "type": "string",
              "value": "Hi, this is your virtual assistant. How can I help you?"
          },
          "send":{
              "type": "string",
              "value": "I want to make a payment"
          }
      },
      {
          "receive":{
              "type": "string",
              "value": "I can help with that. What is your PIN number?"
          },
          "send":{
              "type": "string",
              "value": "1234"
          }
      }
  ]
}

The following table describes more properties for the JSON object:

Table 4. More properties for the JSON object
Properties	Description
id	Specifies the GUID for the test case
rev	Specifies the current version of the test case

To modify an existing test case, use a PUT request like the following example.

Use the returned ID of the test case and pass a modified version of the JSON object that you used to create the test case:
```
PUT /voice-agent-tester/v1/case/<Returned GUID of the test case>
```
To delete a test case, use a DELETE request like the following example.

Use the returned ID of the test case.
```
DELETE /voice-agent-tester/v1/case/<returned GUID of the test case>
```
Get a list of all the test cases in the query namespace by using a GET request like the following example.
```
GET /voice-agent-tester/v1/case?namespace=<your_namespace>
```
Delete all test cases in the query namespace by using a DELETE request like the following example.
```
DELETE /voice-agent-tester/v1/case?namespace=<your_namespace>
```