Installing operators for services that require GPUs

If you plan to install services that require graphics processing units (GPUs), you must install several operators that support the management of the NVIDIA software components that are needed to provision GPUs.

Installation phase
  • You are not here. Setting up a client workstation
  • You are not here. Setting up a cluster
  • You are not here. Collecting required information
  • You are not here. Preparing to run installs in a restricted network
  • You are not here. Preparing to run installs from a private container registry
  • You are here icon. Preparing the cluster for IBM® Software Hub
  • You are not here. Preparing to install an instance of IBM Software Hub
  • You are not here. Installing an instance of IBM Software Hub
  • You are not here. Setting up the control plane
  • You are not here. Installing solutions and services
Who needs to complete this task?

Cluster administrator A cluster administrator must complete this task.

When do you need to complete this task?

One-time setup Complete this task if you plan to install one or more of the following services:

  • IBM Knowledge Catalog Premium *
  • IBM Knowledge Catalog Standard *
  • Watson Machine Learning *
  • Watson Studio Runtimes *
  • watsonx.ai™
  • watsonx Assistant *
  • Watsonx BI
  • watsonx Code Assistant™
  • watsonx Code Assistant for Red Hat® Ansible® Lightspeed
  • watsonx Code Assistant for Z
  • watsonx Code Assistant for Z Agentic 5.2.1 and later
  • watsonx Code Assistant for Z Code Explanation
  • watsonx Code Assistant for Z Code Generation 5.2.1 and later
  • watsonx.data™ *
  • watsonx.data Premium
  • watsonx.data intelligence
  • watsonx™ Orchestrate *

An asterisk (*) indicates that the service requires GPU in some situations.

About this task

Review the following table to determine whether you need to install GPU operators based on the services that you plan to install:

Service GPU operators
IBM Knowledge Catalog Premium
  • Required if you run the models on GPU.
  • Not required if you run the models on:
    • CPU
    • A remote instance of watsonx.ai
IBM Knowledge Catalog Standard
  • Required if you run the models on GPU.
  • Not required if you run the models on:
    • CPU
    • A remote instance of watsonx.ai
Watson Machine Learning

Required only if you want to use deep learning or machine learning models that require GPU.
Watson Studio Runtimes

Required only if you want to use runtimes that require GPU.
watsonx.ai

Always required.
watsonx Assistant

Required only if you want to use features that require GPU.
watsonx BI

Always required.
watsonx Code Assistant

Always required.
watsonx Code Assistant for Red Hat Ansible Lightspeed

Always required.
watsonx Code Assistant for Z

Always required.
watsonx Code Assistant for Z Agentic 5.2.1 and later

Always required.
watsonx Code Assistant for Z Code Explanation

Always required.
watsonx Code Assistant for Z Code Generation 5.2.1 and later

Always required.
watsonx.data

Required only if you want to use Milvus indexes that require GPU.
watsonx.data Premium

Always required.
watsonx.data intelligence

Always required.
watsonx Orchestrate
  • Required if you run models locally.
  • Not required if you use the AI gateway to access third-party models.

Procedure

The steps that you must complete depend on whether your cluster is connected to the internet:

  1. To install the operators on a cluster that can connect to the internet:
    1. Complete Installing the Node Feature Discovery (NFD) Operator in the NVIDIA GPU Operator on Red Hat OpenShift® Container Platform documentation.
    2. Complete Installing the NVIDIA GPU Operator in the NVIDIA GPU Operator on Red Hat OpenShift Container Platform documentation.
  2. To install the operators on a cluster that is disconnected or air-gapped:
    1. Complete Deploy GPU Operators in a disconnected or air-gapped environment in the NVIDIA GPU Operator on Red Hat OpenShift Container Platform documentation.

What to do next

Now that you've installed the operators for services that require GPUs, you're ready to complete Installing Red Hat OpenShift AI.

You can optionally complete Configuring NVIDIA Multi-Instance GPU (MIG) if your environment includes services that support MIG.