Configuring test cases for Voice Agent Tester

You can create a test case to run automated voice tests against a voice agent by creating a JSON object and then posting that object. You can also do more actions, such as modifying or deleting test cases.

A test case contains the outline of a conversation and is the basic resource that is needed for running tests. The JSON object for a single test case includes metadata about the test along with a JSON array called turns, which contains an array of turn objects. Each turn object contains the expected transcribed audio from the voice agent under test. Additionally, each turn object contains a response that is synthesized into audio that is sent back to the voice agent under test. The data model also supports the ability to look for substrings and run regex expressions against the transcribed audio.

Creating a test case

To create a test case, first create a JSON object and then do a POST request for the JSON object.

You can use the following example to create your JSON object:

{
    "name" : "full payment flow",
    "description" : "This test verifies the full payment dialog flow",
    "namespace":"default",
    "turns" : [
        {
            "receive":{
                "type": "string",
                "value": "Hi, this is your virtual assistant. How can I help you?"
            },
            "send":{
                "type": "string",
                "value": "I want to make a payment"
            }
        },
        {
            "receive":{
                "type": "string",
                "value": "I can help with that. What is your PIN number?"
            },
            "send":{
                "type": "string",
                "value": "1234"
            }
        }
    ]
}

The following table describes the properties of the JSON object:

Table 1. Properties for the JSON object
Property Description
name Specifies the name of the test case
description Optionally specifies a more detailed description for the test case
namespace Optionally defines a group to which a test case is bound. If no namespace is specified, the namespace is default.
turns Specifies an ordered array representing a conversation. Each index is a turn in the conversation represented by a JSON object composed of what you expect to receive and what you would like to send.

The following table describes the supported types for the receive JSON object:

If any of the receive objects don't match the response from the voice agent under test, the test case is marked in the final results as a failure.

Table 2. Supported types for the receive JSON object
Type Definition
string Specifies the exact case insensitive string expected from the transcribed audio. This audio is received from the voice agent under test.
substring Specifies a substring that exists within the transcribed audio. This audio is received from the voice agent that is under test.
regex Specifies a regex expression that is run against the transcribed audio. This audio is received from the voice agent under test.
context Specifies a JSON object that contains the property and value to look for in the returned context.
orList Specifies an array of receive objects, any of which can be matched with the input from the voice agent under test.
andList Specifies an array of receive objects, all of which must be matched with the input from the voice agent under test.

The following table describes the supported types for the send JSON object:

Table 3. Supported types for the send JSON object
Type Definition
string Specifies the text to synthesize into audio and stream back to the voice agent under test. For more information, see the action sequences.
commands Specifies a Voice Gateway command sequence to initiate back to the voice agent under test. For more information, see the action sequences

Next, create the test case by entering a POST request for the JSON object.

Use the following example:

POST /voice-agent-tester/v1/case

If the POST request is successful, a 201 CREATED response is returned. This response includes the GUID of the test case that you created. You use the GUID to manage the test case.

Additional requests to manage test cases

The following table describes more properties for the JSON object:

Table 4. More properties for the JSON object
Properties Description
id Specifies the GUID for the test case
rev Specifies the current version of the test case