Installing the Custom Metrics Autoscaler

By default, the Red Hat® OpenShift® Horizontal Pod Autoscaler (HPA) uses resource-based automatic scaling to increase or decrease the number of pods in response to CPU or memory consumption based on the utilization threshold.

If you plan to install services with a dependency on Inference foundation models, you can install Red Hat OpenShift Custom Metrics Autoscaler to enable event-driven scaling on GPU. Event-driven scaling enables the cluster to automatically scale model replicas across existing GPU nodes in response to inferencing requests.

Installation phase
  • You are not here. Setting up a client workstation
  • You are not here. Setting up a cluster
  • You are not here. Collecting required information
  • You are not here. Preparing to run installs in a restricted network
  • You are not here. Preparing to run installs from a private container registry
  • You are here icon. Preparing the cluster for IBM Software Hub
  • You are not here. Preparing to install an instance of IBM Software Hub
  • You are not here. Installing an instance of IBM Software Hub
  • You are not here. Setting up the control plane
  • You are not here. Installing solutions and services
Who needs to complete this task?
Cluster administrator A cluster administrator must complete this task.
When do you need to complete this task?

One-time setup This task is optional but recommended if you plan to install one or more services that use Inference foundation models to start and host foundation models:

  • IBM Knowledge Catalog Premium *
  • IBM Knowledge Catalog Standard *
  • Watson Speech services *
  • watsonx.ai™
  • watsonx Assistant *
  • Watsonx BI
  • watsonx Code Assistant™
  • watsonx Code Assistant for Red Hat Ansible® Lightspeed
  • watsonx Code Assistant for Z Agentic
  • watsonx Code Assistant for Z Understand
  • watsonx.data™ Premium
  • watsonx.data integration *
  • watsonx.data intelligence *
  • watsonx™ Orchestrate *

About this task

The Red Hat OpenShift Custom Metrics Autoscaler automatically installs the Kubernetes Event-driven Autoscaler (KEDA).

With KEDA, the cluster can scale the number of model replicas in response to the inferencing load.
  • If there are no new inferencing requests, the cluster scales down the number of model replicas to free up GPU.
  • If the number of inferencing requests increases, the cluster scales up the number of model replicas on available GPU to maintain latency.
Important: To use event-driven scaling with an instance of IBM Software Hub, you must also:
  1. Enable event-driven automatic scaling on GPUs for the instance.
  2. Configure event-driven scaling for models that are running in the instance.

Procedure

  1. Ensure that user-defined workload monitoring is enabled.
    For more information, see Enabling monitoring for user-defined projects in the Red Hat OpenShift Container Platform documentation:
  2. Install the Custom Metrics Autoscaler from the OpenShift Console.
    For more information, see Installing the Custom Metrics Autoscaler in the Red Hat OpenShift Container Platform documentation:

What to do next

Now that you've installed Custom Metrics Autoscaler, you're ready to complete Installing and setting up Multicloud Object Gateway for IBM Software Hub.