Upgrading IBM watsonx.ai from Version 4.8 to Version 5.1

An instance administrator can upgrade IBM watsonx.ai from IBM Cloud Pak® for Data Version 4.8 to IBM Software Hub Version 5.1.

Who needs to complete this task?

Instance administrator To upgrade IBM watsonx.ai, you must be an instance administrator. An instance administrator has permission to manage software in the following projects:

The operators project for the instance: The operators for this instance of IBM watsonx.ai are installed in the operators project. In the upgrade commands, the ${PROJECT_CPD_INST_OPERATORS} environment variable refers to the operators project.
The operands project for the instance: The custom resources for the control plane and IBM watsonx.ai are installed in the operands project. In the upgrade commands, the ${PROJECT_CPD_INST_OPERANDS} environment variable refers to the operands project.

When do you need to complete this task?

Review the following options to determine whether you need to complete this task:

If you want to upgrade the IBM Software Hub control plane and one or more services at the same time, follow the process in Upgrading an instance of IBM Software Hub instead.
If you didn't upgrade IBM watsonx.ai when you upgraded the IBM Software Hub control plane, complete this task to upgrade IBM watsonx.ai.
Repeat as needed If you are responsible for multiple instances of IBM Software Hub, you can repeat this task to upgrade more instances of IBM watsonx.ai on the cluster.

Information you need to complete this task

Review the following information before you upgrade IBM watsonx.ai:

Version requirements: All the components that are associated with an instance of IBM Software Hub must be installed at the same release. For example, if the IBM Software Hub control plane is at Version 5.1.3, you must upgrade IBM watsonx.ai to Version 5.1.3.

Environment variables

The commands in this task use environment variables so that you can run the commands exactly as written.

If you do not have the script that defines the environment variables, see Setting up installation environment variables.
To use the environment variables from the script, you must source the environment variables before you run the commands in this task. For example, run:
```
source ./cpd_vars.sh
```

Common core services: IBM watsonx.ai requires the IBM Software Hub common core services.
If the common core services are not at the correct version in the operands project for the instance, the common core services are automatically upgraded when you upgrade IBM watsonx.ai. The common core services upgrade increases the amount of time the upgrade takes to complete.

Adding compute resources

Add extra GPU capacity to the worker nodes before you upgrade the software. The newest foundation models are added first, and then the resources assigned to the original versions are released. For a nondisruptive upgrade, make sure that your worker nodes have additional GPU capacity to temporarily handle both the newer deployments and the original deployments. To determine how many GPUs are needed, complete the following steps:

Find out what models are deployed in the cluster now. To do so, run the following command:

oc describe watsonxaiifm -n ${PROJECT_CPD_INST_OPERANDS}

Check the section that lists the installed foundation models. For example, the following excerpt shows a deployment with many models installed:

spec:  
  install_model_list:
    ibm-granite-13b-instruct-v1
    ibm-granite-13b-chat-v1
    bigscience-mt0-xxl
    google-flan-ul2
    google-flan-t5-xl
    google-flan-t5-xxl
    meta-llama-llama-2-13b-chat

If llama-2-70b-chat is installed, 5 extra GPUs are needed on the worker node. Otherwise, you need 2 extra GPUs.

If you can't add more GPUs to the cluster, you can scale down an existing model deployment to make room for the pods that come up during the upgrade. To scale down a model deployment, use the following command:

oc scale deployment -n ${PROJECT_CPD_INST_OPERANDS} \
{model-id}-inference-server --replicas=0

where {model-id} is the name of the model that you want to scale down. For example, to scale down the granite-13b-instruct-v1 model to make room for the new version, you can use the following command:

oc scale deployment -n ${PROJECT_CPD_INST_OPERANDS} \
ibm-granite-13b-instruct-v1-inference-server --replicas=0

Before you begin

This task assumes that the following prerequisites are met:

Prerequisite	Where to find more information
The cluster meets the minimum requirements for IBM watsonx.ai.	If this task is not complete, see System requirements.
The workstation from which you will run the upgrade is set up as a client workstation and has the following command-line interfaces: IBM Software Hub CLI: `cpd-cli` OpenShift® CLI: `oc`	If this task is not complete, see Updating client workstations.
The IBM Software Hub control plane is upgraded.	If this task is not complete, see Upgrading an instance of IBM Software Hub.
For environments that use a private container registry, such as air-gapped environments, the IBM watsonx.ai software images are mirrored to the private container registry.	If this task is not complete, see Mirroring images to a private container registry.
For environments that use a private container registry, such as air-gapped environments, the `cpd-cli` is configured to pull the `olm-utils-v3` image from the private container registry.	If this task is not complete, see Pulling the olm-utils-v3 image from the private container registry.
The operators required to use GPUs are installed.	If this task is not complete, see Installing operators for services that require GPUs.
Red Hat® OpenShift AI is installed.	If this task is not complete, see Installing Red Hat OpenShift AI.

Procedure

Complete the following tasks to upgrade IBM watsonx.ai:

Before upgrading
Upgrading the service
Validating the upgrade
What to do next

Before upgrading

If you are upgrading and using models with the watsonx-cfm-caikit-1.0 software specification, especially with text chat, this upgrade will fail.

Text chat functionality is only available on models with the watsonx-cfm-caikit-1.1 software specification, which uses the VLLM image.

Note: The watsonx-cfm-caikit-1.1 software specification is only available on new deployments. You can not upgrade from an existing deployment.

Upgrading the service

Important: The Operator Lifecycle Manager (OLM) objects for IBM watsonx.ai were updated when you upgraded the IBM Software Hub platform. The

cpd-cli
manage
apply-olm

updates all of the OLM objects in the operators project at the same time.

To upgrade IBM watsonx.ai:

Log the cpd-cli in to the Red Hat OpenShift Container Platform cluster:
```
${CPDM_OC_LOGIN}
```
Remember: CPDM_OC_LOGIN is an alias for the cpd-cli manage login-to-ocp command.

Update the custom resource for IBM watsonx.ai.

cpd-cli manage apply-cr \
--components=watsonx_ai \
--release=${VERSION} \
--cpd_instance_ns=${PROJECT_CPD_INST_OPERANDS} \
--license_acceptance=true \
--upgrade=true

Validating the upgrade

IBM watsonx.ai is upgraded when the apply-cr command returns:

[SUCCESS]... The apply-cr command ran successfully

If you want to confirm that the custom resource status is Completed, you can run the cpd-cli manage get-cr-status command:

cpd-cli manage get-cr-status \
--cpd_instance_ns=${PROJECT_CPD_INST_OPERANDS} \
--components=watsonx_ai

What to do next

After upgrading the IBM watsonx.ai component, you can now install any new models. For more information, see Adding foundation models to IBM watsonx.ai.

Note:

A modification was made to the granite-13b-chat-v2 foundation model with the 4.8.4 release.
A modification was made to the granite-20b-multilingual foundation model with the 5.0.0 release.
A modification was made to the granite-3b-code-instruct foundation model with the 5.1.0 release.

If you added any of these models to your deployment previously, the modification is applied automatically when you upgrade the service.