Contribute in GitHub:

Programming agent assistants and SIPREC

When you configure Voice Gateway to support agent assistants, media from the call must be shared with Voice Gateway for call audio transcription and forwarding to connected services.

Voice Gateway acts as a SIPREC Session Recording Server (SRS) when configured as an agent assistant. You can fork calls from any SIPREC Session Recording Client (SRC), such as a Session Border Controller (SBC), to Voice Gateway to handle real-time transcriptions. These transcriptions are then forwarded as a turn event to a Service Orchestration Engine (SOE) or a reporting event to a reporting REST server or IBM Cloudant service.

When a SIPREC session is opened with Voice Gateway, it receives separate audio streams for both the caller and the agent from the SRC. Voice Gateway transcodes and forwards the call audio to a speech-to-text processing engine, and then sends the returned transcriptions to any of the configured services.

Voice Gateway also handles parsing of the SIPREC metadata to associate source participant details for each transcribed utterance along with the related parsed session details.

Connecting to a Service Orchestration Engine

Voice Gateway communicates with Watson Assistant / Service Orchestration Engine (SOE) through the Watson Assistant REST API, sending request data and receiving a corresponding JSON response. For the REST request / response messages format, refer to the Watson Assistant REST API documentation.

When using the Watson Assistant API v1, a Service Orchestration Engine has to provide the mandatory conversation_id field in all response messages.

{
  "context": {
    "conversation_id": "xxxx"
  }
}

Publishing transcriptions to an SOE with the Watson Assistant API

In an agent assistant environment, you can configure Voice Gateway to send transcriptions to an SOE through the Watson Assistant REST API. A transcription is sent in the input.text field.

Remember: With SIPREC, the streams are uni-directional into Voice Gateway only.

Values for the following state variables are extracted from the SIPREC metadata and sent to the SOE with each transcription or utterance.

Table 1. State variables extracted from SIPREC metadata.
State variable	SIPREC specific description
`vgwSessionID`	Set to `gcid` by default. Also supports `callid`, `ucid`, and `session_id`. See State Variables.
`vgwSIPCallID`	For SIPREC, Voice Gateway will attempt to pull this from the SIPREC metadata. If no SIP call ID is found in the SIPREC metadata, it inserts the SIP call ID from the SIPREC SIP INVITE. See State Variables.
`vgwSIPToURI`	Represents the callees address of record. Must be configured. See State Variables.
`vgwSIPFromURI`	Represents the callers address of record. See State Variables.

SIPREC related state variables that Voice Gateway sends to the SOE.

Table 2. SIPREC related state variables sent to the SOE.
State variable	SIPREC specific description
`vgwTranscriptionSource`	State variable that contains the participant Address Of Record (AOR) where the transcription came from. Version 1.0.0.7a and later.
`vgwIsCaller`	State variable set to `Yes` for caller and `No` for agent. Set to `No` by default. Version 1.0.0.7a and later.
`vgwSTTResponse`	Contains the final response received from the Speech to Text service.
`vgwSIPRequestURI`	Pulled from the SIPREC SIP INVITE request URI. See State Variables.
`vgwSIPCustomInviteHeader`	Header pulled from the SIPREC INVITE message, if configured. See State Variables.
`vgwSIPCustomInviteHeaders`	Headers pulled from the SIPREC INVITE message, if configured. See State Variables.
`vgwIsDTMF`	Set to `Yes` when utterance DTMF input.
`vgwNetworkWarnings`	When RTCP is enabled, Voice Gateway sends network warnings to the configured SOE.

SIPREC related state variables that the SOE sends to the Voice Gateway.

Table 3. SIPREC related state variables sent to the Voice Gateway
State variable	SIPREC specific description
`vgwPostResponseTimeoutCount`	The amount of time, in milliseconds, to wait for an input from the participants. If this value is exceeded, the Watson Assistant receives a text update with the word `vgwPostResponseTimeout` to indicate that a timeout occurred. By default, the response timer isn't supported in SIPREC sessions. The SOE must set this state variable to enable the response timeout. Version 1.0.1.0 and later.

Reporting events

An alternate way to publish transcription events is to a reporting server. This interface can also be used to store transcription events to IBM Cloudant or a reporting REST server, such as Splunk.

See Reporting transcription events from Voice Gateway for more details.

Configuring SIPREC connections

You can configure your SIPREC connections to send transcription events at an SOE or a reporting REST server, or both.

Configuring an SOE to receive transcription events

To configure your SOE to receive transcription events through the Watson Assistant REST API from Voice Gateway with Docker environment variables, add the following environment variables to your Voice Gateway deployment file.

Table 4. Environment variables to configure SOE to receive transcription events
Environment Variable	description
`WATSON_CONVERSATION_URL`	See SIP Orchestrator environment variables.
`WATSON_CONVERSATION_USERNAME`	Configure if SOE requires a basic authentication user name. See SIP Orchestrator environment variables.
`WATSON_CONVERSATION_PASSWORD`	Configure if SOE requires a basic authentication password. See SIP Orchestrator environment variables.

You can also use JSON properties to configure your SOE. The following code example shows how to configure the conversation object to include the SOE configuration. You use your SOE url, password, and username instead of the credentials for a Watson Assistant instance.

{
  "tenantURI": "siprec-tenant",
  "conversation": {
    "url": "https://your-soe.com/soe-api",
    "password": "soe-defined-password",
    "username": "soe-defined-username"
  },
}

Configuring SIPREC with Docker environment variables

Use the following environment variables when you set up Voice Gateway for agent assistants. See Configuration environment variables for Voice Gateway.

Table 5. Environment variables to configure Voice Gateway for agent assistants.
Environment Variable	Value	SIPREC specific description
`CUSTOM_SIPREC_SESSION_FIELD`	None	Set to `gcid` by default. Also supports `callid`, `ucid`, and `session_id`. See SIP Orchestrator environment variables.
`SEND_SIP_TO_URI_TO_CONVERSATION`	`true`/`false`	Represents the callees address of record. See SIP Orchestrator environment variables.
`SEND_SIP_FROM_URI_TO_CONVERSATION`	`true`/`false`	The callers address of record. See SIP Orchestrator environment variables.
`SEND_SIP_CALL_ID_TO_CONVERSATION`	`true`/`false`	For SIPREC, Voice Gateway attempts to pull this from the SIPREC metadata. If no SIP call ID is found, it will insert the SIP call ID from the SIPREC SIP INVITE.
`CUSTOM_SIP_INVITE_HEADER`	None	See SIP Orchestrator environment variables.
`CUSTOM_SIP_SESSION_HEADER`	None	Used for `vgwSessionID` if the CUSTOM_SIPREC_SESSION_FIELD isn't found in the SIPREC metadata. See SIP Orchestrator environment variables.
`WHITELIST_TO_URI`	None	See SIP Orchestrator environment variables. For SIPREC, the environment variable is supported in Version 1.0.2 and later.
`WHITELIST_SIPREC_ATTR_NAME`	None	See SIP Orchestrator environment variables.
`WHITELIST_SIPREC_ATTR_VALUE`	None	See SIP Orchestrator environment variables.

When a new call comes in, Voice Gateway introspects the SIPREC metadata and looks for a WHITELIST_SIPREC_ATTR_NAME and WHITELIST_SIPREC_ATTR_VALUE pair. If no match is found, Voice Gateway introspects the SIP To URI or SIP Request URI to verify if it matches the value specified in WHITELIST_TO_URI.

Configuring SIPREC with a multi-tenant JSON configuration file

In a multi-tenant deployment, configuration settings for SIPREC sessions are specified in a tenant by using a siprec-tenant ID for the tenantURI property. See Advanced JSON configuration properties.

The following example shows a configuration for publishing transcriptions to an SOE by using the Watson Assistant REST API.

{
  "tenantURI": "siprec-tenant",
  "description": "SIPREC tenant",
  "sendSIPCallIDToConversation": true,
  "sendSIPToUriToConversation": true,
  "sendSIPFromUriToConversation": true,
  "whitelistSiprecAttrName": "aor",
  "whitelistSiprecAttrValue": "123456789",
  "customSIPRECSessionField": "ucid",
  "conversation": {
    "url": "https://your-soe.com/soe-api",
    "password": "soe-defined-password",
    "username": "soe-defined-username"
  },
  "stt": {
    "credentials": {
      "apikey": "{apikey}",
      "tokenServiceProviderUrl": "https://iam.cloud.ibm.com/identity/token"
    },
    "config": {
      "profanity_filter": true,
      "smart_formatting": true,
      "model": "en-US_NarrowbandModel"
    }
  }
}

Table 6. JSON properties to configure Voice Gateway for agent assistants.
JSON Property	Value	SIPREC specific description
`customSIPRECSessionField`	None	Set to `gcid` by default. Also supports `callid`, `ucid`, and `session_id`. See Optional tenant properties.
`sendSIPToUriToConversation`	`true`/`false`	Represents the callees address of record. See Optional tenant properties.
`sendSIPFromUriToConversation`	`true`/`false`	The callers address of record. See Optional tenant properties.
`sendSIPCallIDToConversation`	`true`/`false`	For SIPREC, Voice Gateway attempts to pull this from the SIPREC metadata. If no SIP call ID is found, it will insert the SIP call ID from the SIPREC SIP INVITE.
`customSIPInviteHeader`	None	See Optional tenant properties.
`customSIPInviteHeaders`	None	See Optional tenant properties.
`customSIPRECSessionField`	None	Used for `vgwSessionID` if the customSIPRECSessionField isn't found in the SIPREC metadata. See Optional tenant properties.
whitelistToUri	None	See Optional tenant properties.
whitelistSiprecAttrName	None	See Optional tenant properties.
whitelistSiprecAttrValue	None	See Optional tenant properties.

Table 7. Actions supported in SIPREC sessions.
JSON Property	SIPREC specific description
`vgwActSetSTTConfig`	See Action tags and state variables in the Voice Gateway API.
`vgwActPauseSTT`	See Action tags and state variables in the Voice Gateway API.
`vgwActPauseDTMF`	See Action tags and state variables in the Voice Gateway API.
`vgwActUnPauseDTMF`	See Action tags and state variables in the Voice Gateway API.
`vgwActAddCustomCDRData`	See Action tags and state variables in the Voice Gateway API.
`vgwActCollectDTMF`	See Action tags and state variables in the Voice Gateway API and Collecting dual-tone multi-frequency (DTMF) digits.
`vgwActHangup`	See Action tags and state variables in the Voice Gateway API.

In some actions, you can use the participants attribute to specify a list of participants that the action should be applied to.