IBM Model Gateway

The IBM Model Gateway provides an interface for managing and routing requests to large language models (LLM).

When installed the Model Gateway orchestrates AI workloads for CP4BA deployments that include Business Automation Workflow. It collects metrics and traces, and can integrate with Instana to display comprehensive status reports.

The gateway is designed for security, on least-privilege principles. For example, it uses non-root container execution, read-only file systems, and no privilege escalations.

Installation

For more information about installing the Model Gateway for your CP4BA production deployments, see Installing IBM Model Gateway.

The installation uses the following resources:

  • A bash script (cert-kubernetes/scripts/model-gateway/cp4a-model-gateway-deployment.sh), which is used to automate the lifecycle management of the Model Gateway operator.
  • A Helm chart (cert-kubernetes/descriptors/CP4BA/helm-charts/model-gateway/) for namespace-scoped CP4BA deployments, which is used to install the Model Gateway operator.
  • A Helm chart (cert-kubernetes/descriptors/CP4BA/helm-charts/model-gateway-cluster-scoped/) for separated CP4BA deployments, which is used to install the cluster-wide resources (CRD and ClusterRole).

Separation of duties

The bash script handles both namespace-scoped and separated deployment methods.

  • In a namespace-scoped deployment, the operators and CP4BA instance are in the same namespace.
  • In a separated deployment (separation of duties), the operators are in a dedicated namespace and the instance is in a separate namespace.

Multi-tenancy

The Helm chart supports tethered namespaces for multi-tenant deployments where a single operator manages the Model Gateway instances across multiple namespaces.