Managing remote models in AI Optimizer for IBM Z and IBM LinuxONE

Register and connect to remote LLM providers such as watsonx.ai™, OpenAI, and Amazon Bedrock to make remotely deployed models available for inference requests. Tech Preview

1. Obtain an authentication token from the IBM Z® Appliance Container Infrastructure (zACI) system API.

   An authentication token is a short-lived credential that is usually valid for 30 minutes. You will use the token to register providers and models.

   Submit an HTTPS POST API request by running the following curl command:

   token=$(curl -k -X POST https://<ssc-lpar-ip>:443/api/com.ibm.zaci.system/api-tokens \
      -H "zACI-API: com.ibm.zaci.system/1.0" \
      -H "Accept: application/vnd.ibm.zaci.payload+json;version=1.0" \
      -H "Content-Type: application/vnd.ibm.zaci.payload+json;version=1.0" \
      -d '{
            "kind": "request",
            "parameters": {
              "user": "<username>",
              "password": "<password>"
              }
           }' | jq -r '.parameters.token')
   
   echo $token

   Where:

   • <ssc-lpar-ip> is the IP address of your AI Optimizer for Z and LinuxONE SSC LPAR.
   • <username> is your username for the Appliance Manager UI.
   • <password> is your password for the Appliance Manager UI.

   The authentication token is stored in the token variable and displayed in the output.

2. Register a provider instance.

   Submit an HTTPS POST API request by running the following curl command:

   curl 'https://<lpar_ip>/aioui/api/proxy?key=providersInfo' \
     -X POST \
     -H 'Content-Type: application/json' \
     -H 'Accept: application/json' \
     -H 'Cookie: auth_token=<auth_token>' \
     --data '{
       "name": "<provider_instance_name>",
       "type": "<provider_type>",
       "data": {
         <provider_specific_configuration>
       }
     }'

   Where:

   • <lpar_url> is the URL of your LPAR.
   • <auth_token> is the authentication token obtained in Step 1.
   • <provider_instance_name> is a name for the provider instance.
   • <provider_type> is the provider type (for example, watsonxai, openai, bedrock, anthropic, or mistral).
   • <provider_specific_configuration> represents the configuration parameters required by the selected provider.

   Note: The structure and required fields in the data section vary depending on the provider type. These fields might include API keys, endpoints, project or space identifiers, or other provider-specific settings. For complete API specifications, including supported provider types and their configuration schemas, see the Remote model provider API reference.

   The provider instance is created and can be used when registering remote models.

3. Register a model under the provider instance.

   Submit an HTTPS POST API request by running the following curl command:

   curl 'https://<lpar_url>/aioui/api/proxy?key=llmsInfo' \
     -X POST \
     -H 'Content-Type: application/json' \
     -H 'Accept: application/json' \
     -H 'Cookie: auth_token=<auth_token>' \
     --data '{
       "name": "<model_display_name>",
       "type": "<model_identifier>",
       "modelProviderName": "<provider_instance_name>",
       "tags": [
         {
           "name": "remote"
         }
       ]
     }'

   Where:

   • <lpar_url> is the URL of your LPAR.
   • <auth_token> is the authentication token obtained in Step 1.
   • <model_display_name> is a display name for the model.
   • <model_identifier> is the model identifier used by the remote provider.
   • <provider_instance_name> is the name of the provider instance that hosts the model.

   The model is registered and associated with the selected provider. It is now available for remote inference requests.

   Note:

   When you send inference requests to remote providers, request body parameters might be handled differently depending on the provider type.

   • Some providers (for example, OpenAI, Anthropic, and Amazon Bedrock) forward the request body unchanged to the remote service.
   • Other providers (such as IBM watsonx.ai and Google Gemini) might modify or map request parameters before forwarding them, and might drop unsupported parameters.
   • In some cases, provider-specific configuration values (such as project or space IDs) might be applied during request processing.
   • Even when parameters are forwarded unchanged, the remote provider might ignore, modify, or reject unsupported parameters.
4. Optional: Manage model tags through the UI.

   After the model is registered, you can add or modify tags through the AI Optimizer for Z and LinuxONE UI. Tags enable the registered models to be used for routing purposes, allowing you to categorize and organize models for different use cases.