Azure openai-invoke

Use the Azure OpenAI invoke policy to invoke Azure OpenAI operations.

Gateway support

Table 1. Table showing which gateways support this policy, and the corresponding policy version
Gateway	Policy version
DataPower® API Gateway	2.0.0

This topic describes how to configure the policy in your OpenAPI source; for details on how to configure the policy in the assembly user interface, see Azure openai invoke.

About

The Azure OpenAI invoke policy provides access to four Azure OpenAI operations:

POST /chat/completions
POST /embeddings
GET /models
GET /models/[model-id]

Properties

The following table lists the policy properties, indicates whether a property is required, specifies the valid and default values for input, and specifies the data type of the values.

Table 2. Azure OpenAI invoke policy properties
Property name	Required	Description	Data type
version	Yes	The policy version number. You must specify a version for the policy that is compatible with the gateway that you are using. When the API is published, if the version is incompatible with the gateway, a validation error is thrown that specifies the available versions.	string
title	No	The title of the policy; the default value is Azure OpenAI invoke chat completions.	string
description	No	A description of the policy.	string
operation	Yes	The Azure OpenAI operation to invoke; must be one of the following operations: `POST /chat/completions` An HTTP POST operation for `/chat/completions`. See Create chat completion in the Azure OpenAI documentation for sample request and response examples. `POST embeddings` An HTTP POST operation for `/embeddings`. See Create embeddings in the Azure OpenAI documentation for sample request and response examples. `GET models` An HTTP GET operation for `/models`. See List Models in the OpenAI documentation for sample response examples. `GET models/[model]` An HTTP GET operation for `/models/[model]`. See Retrieve model in the Azure OpenAI documentation for sample response examples.	string
api_key	Yes	The API key, or a variable reference to the variable that contains an API key. The API key is used to authenticate Azure OpenAI requests.	string
deployment_id	Yes	Name of project/deployment in Azure where model is deployed.	string
resource_id	Yes	Serves as part of endpoint of URL that points to your provisioned resources.	string
enable_cache	No	Response caching is enabled by default to optimize API performance; however, you can disable it if needed for your API. When response caching is enabled, then when a request is sent to the OpenAI service, the response cache is inspected to determine if the request payload has an associated cached response. If so, that cached response and its associated HTTP response headers are placed into theDataPower API Gateway context message specified by the output property. If there is no cached response, the request is passed to the OpenAI service, and the response is cached for subsequent operations using the time-to-live specified in the cache_ttl property.	boolean
cache_ttl	No	The duration of the cache (the time-to-live). The minimum duration is 60 seconds and the maximum duration is 86400 seconds (1 day) -- any value outside of that range will fail validation when the API is published, even if the value is accepted in this field.	integer
cache_scope	No	A string that adds uniqueness to the key used to store a cached response. Adding a cache scope enables the same request to be saved in multiple cache entries.	string
op-version	No	The operation version specifies the version of the OpenAI operation (such as completions, chat/completions, embeddings) that the policy applies to. It is defined using YYYY-MM-DD format.
path_params	No	For operations that have a path parameter designated with a path parameter name in square brackets in the operation, specify the name of the parameter and the value separated by a colon. If the operation has more that one path parameter, each path parameter designation should be separated with a comma. For example, for operation `GET models/[model]`, the path parameter property would be specified as `model:somename`. Note: If a path parameter is required for the operation selected but was not provided, the invoke operation will fail. However, for operations that don't require a path parameter, any provided path parameter will be ignored.	string
query_params	No	For operations that support query parameters, specify a query parameter string that specifies the name of the parameter and value separated with an equal sign character. If more than one query parameter is supported, each query parameter would be separated with an ampersand character. The supported operations at this time do not support query parameters, although operations that could be supported in the future would support query parameters.	string
output	No	The name of the DataPower API Gateway context message that receives the OpenAI response. By default, this property uses `message` as the value, which enables the response to be accessible by subsequent API policies in `message.body` and `message.headers`.	string

Example


azure-openai-invoke:
          title: Azure-OpenAI invoke policy
          description: Azure-OpenAI invoke policy desc
          api_key: keyvalue
          deployment_id: $(dep-id)
          resource_id: $(res-id)
          operation: POST chat/completions
          op-version: $(azure-op-version)
          path_params:  
          enable_cache: true
          cache_ttl: 60
          cache_scope: azureScope
          output: request
          version: 2.0.0