Using the AI Gateway to support watsonx.ai APIs

The API Connect provides a UI wizard to create AI-aware APIs and products, plus integration with watsonx.ai to forward requests and manage responses.

The AI Gateway makes it easy for enterprises to manage access to API endpoints used by AI applications. The AI Gateway simplifies the integration of AI into new and existing OpenAPI 3.0 APIs in API Connect to access a set of operations exposed by watsonx.ai.

API Connect provides a policy that enables your API to send requests to watsonx.ai. The following watsonx.ai specs are supported:
  • /text-generate
  • /tokenize
  • /deployment-models

Prerequisites for using the AI Gateway with watsonx.ai

Before attempting to use the AI Gateway, complete the following prerequisites:

  • Obtain an IBM Cloud API key.

    When you register for an IBM Cloud ID, an API key is automatically assigned. You can create additional API keys if needed. An API key is used by the IBM Cloud Authentication user defined policy to generate a Bearer token, which is used to authenticate for the desired watsonx.ai service. For information on managing API key, see Understanding API keys in the IBM Cloud documentation.

  • Create a catalog property named watsonx-ai-apikey and set its value to the actual API key required for accessing watsonx.ai services. For information on creating catalog specific property values, see Defining Catalog specific property values.
  • Create a watsonx.ai project.

    A watsonx.ai project is required for the user that is identified by the API key. The project will have a project ID. For information on creating projects, see Creating a project in the watsonx.ai documentation.

  • Link the watsonx.ai project to an LLM service by completing the following steps:
    1. In the project, click Services and Integrations in the navigation list.
    2. Create a service; for example, you might call it "Watson Machine Learning".
    3. Associate that service with your project.

    Failure to configure your project correctly will result in a 400 Bad Request response for attempted watsonx.ai operations.

Getting started with the AI Gateway and watsonx.ai

To use the AI Gateway with watsonx.ai, complete the following steps:

  1. Set up your environment as explained in Using the AI Gateway to support watsonx.ai APIs.

  2. Create an API to use as a reverse-proxy.

  3. Add the IBM Cloud Authentication and watsonx.ai invoke policies to the API so it can authenticate with IBM Cloud and access watsonx.ai.

  4. Review metrics on the API's performance in the AI usage dashboard, which tracks AI token and model usage.