Upgrading watsonx Assistant from Version 4.8 to Version 5.1
An instance administrator can upgrade watsonx Assistant from IBM Cloud Pak® for Data Version 4.8 to IBM Software Hub Version 5.1.
- Who needs to complete this task?
-
Instance administrator To upgrade watsonx Assistant, you must be an instance administrator. An instance administrator has permission to manage software in the following projects:
- The operators project for the instance
-
The operators for this instance of watsonx Assistant are installed in the operators project. In the upgrade commands, the
${PROJECT_CPD_INST_OPERATORS}environment variable refers to the operators project. - The operands project for the instance
-
The custom resources for the control plane and watsonx Assistant are installed in the operands project. In the upgrade commands, the
${PROJECT_CPD_INST_OPERANDS}environment variable refers to the operands project.
- When do you need to complete this task?
-
Review the following options to determine whether you need to complete this task:
- If you want to upgrade the IBM Software Hub control plane and one or more services at the same time, follow the process in Upgrading an instance of IBM Software Hub instead.
- If you didn't upgrade watsonx Assistant when you upgraded the IBM Software Hub
control plane, complete this task to upgrade watsonx Assistant.
Repeat as needed If you are responsible for multiple instances of IBM Software Hub, you can repeat this task to upgrade more instances of watsonx Assistant on the cluster.
Information you need to complete this task
Review the following information before you upgrade watsonx Assistant:
- Version requirements
-
All the components that are associated with an instance of IBM Software Hub must be installed at the same release. For example, if the IBM Software Hub control plane is at Version 5.1.3, you must upgrade watsonx Assistant to Version 5.1.3.
- Environment variables
- The commands in this task use environment variables so that you can run the commands exactly as
written.
- If you do not have the script that defines the environment variables, see Setting up installation environment variables.
- To use the environment variables from the script, you must source the environment variables
before you run the commands in this task. For example,
run:
source ./cpd_vars.sh
- Common core services
- watsonx Assistant requires the IBM Software Hub
common core services.
If the common core services are not at the correct version in the operands project for the instance, the common core services are automatically upgraded when you upgrade watsonx Assistant. The common core services upgrade increases the amount of time the upgrade takes to complete.
Before you begin
This task assumes that the following prerequisites are met:
| Prerequisite | Where to find more information |
|---|---|
| The cluster meets the minimum requirements for watsonx Assistant. | If this task is not complete, see System requirements. |
The workstation from which you will run the upgrade is set up as a client workstation and
has the following command-line interfaces:
|
If this task is not complete, see Updating client workstations. |
| The IBM Software Hub control plane is upgraded. | If this task is not complete, see Upgrading an instance of IBM Software Hub. |
| For environments that use a private container registry, such as air-gapped environments, the watsonx Assistant software images are mirrored to the private container registry. | If this task is not complete, see Mirroring images to a private container registry. |
For environments that use a private container registry, such as air-gapped environments,
the cpd-cli is configured to pull the olm-utils-v3 image from the private container registry. |
If this task is not complete, see Pulling the olm-utils-v3 image from the private container registry. |
| If you plan to use features that require GPUs, the operators required to use GPUs are installed. | If this task is not complete, see Installing operators for services that require GPUs. |
| Red Hat® OpenShift AI is installed. | If this task is not complete, see Installing Red Hat OpenShift AI. |
| Multicloud Object Gateway is upgraded, if needed. | If this task is not complete, see Upgrading Multicloud Object Gateway. |
| Red Hat OpenShift Serverless Knative Eventing is upgraded, if needed. | If this task is not complete, see Installing Red Hat OpenShift Serverless Knative Eventing. |
Procedure
Complete the following tasks to upgrade watsonx Assistant:
- Back up your data.
For more information, see Backing up your data in IBM Software Hub in the watsonx Assistant product documentation.
- Backing up and retaining temporary patches
- Specifying additional installation options
- Upgrading the service
- Validating the upgrade
- Validating the EDB Postgres upgrade
- Cleaning up resources
- Upgrading existing service instances
- What to do next
Specifying installation options
When you upgrade watsonx Assistant, specify the following option in the install-options.yml file in the work directory.
################################################################################
# watsonx Assistant parameters
################################################################################
#watson_assistant_size: Production
#watson_assistant_bigpv: false
#watson_assistant_analytics_enabled: true
#watson_assistant_watsonx_ai_type: embedded
#watson_assistant_syom_models: []
#watson_assistant_ootb_models: []
| Property | Description |
|---|---|
watson_assistant_size |
The deployment size for watsonx Assistant. The deployment size determines the number of resources allocated to watsonx Assistant.
|
watson_assistant_bigpv |
Specify whether to create larger physical volumes to improve IOPS performance. Create larger physical volumes if your storage class IOPS performance depends on the size of the physical volume. Important: You cannot change this setting after you install watsonx Assistant.
You do not need to create larger physical volumes if you use the following storage:
|
watson_assistant_analytics_enabled |
Specify whether to store chat logs and analytics.
|
watson_assistant_watsonx_ai_type |
Specify this option if you want to install Inference foundation models (watsonx_ai_ifm) to enable the following features, which require GPUs:
Omit this option if you do not want to enable the preceding features. For more information about supported GPUs, see the Hardware requirements for watsonx Assistant.
|
watson_assistant_syom_models |
5.1.2 and later This option is
available starting in IBM Software Hub
Version 5.1.2. Specify whether you want to use a specialized model
that is specifically tuned for use with watsonx Assistant for:
Important: The following models will be automatically installed
if you install Inference foundation models
(
) and
you do not specify a value for watson_assistant_syom_models or
watson_assistant_ootb_models:
|
watson_assistant_ootb_models |
5.1.2 and later This option is
available starting in IBM Software Hub
Version 5.1.2. Specify whether you want to use a general model for:
Important: The following models will be automatically installed
if you install Inference foundation models
(
) and
you do not specify a value for watson_assistant_syom_models or
watson_assistant_ootb_models:
|
Backing up and retaining temporary patches
Upgrading watsonx Assistant removes all pre-existing temporary patches that do not have the
label 'type=critical-configuration'. If you need to retain some temporary patches,
you can preserve them before upgrading by labeling them as critical. For example, you might have a
custom configuration that is specific to your environment and was not included in the new
release.
- Ignore this section if you don't find any pre-existing temporary patches.
- If you are unsure about what patches need to be retained, contact IBM Support.
To back up and label the patches that need to be retained, follow these steps:
-
Run the following command to list all temporary patches for your instances. Save the list to the file patches_list.txt.
oc get temporarypatch -n ${PROJECT_CPD_INST_OPERANDS} >patches_list.txt -
Save the YAML file for your existing temporary patches by running the following command:
oc get temporarypatch -o yaml > old_patch_backups.yaml - Get the list of the temporary patches from your deployment to determine which patches you want
to
preserve.
oc get temporarypatch -n ${PROJECT_CPD_INST_OPERANDS} - Apply or remove the
"type": "critical-configuration"label as appropriate:- For all patches that you want to retain, use the following
command:
oc label temporarypatch <patch_name> type=critical-configurationFor example:
oc label temporarypatch wa-store-assistant-limits type=critical-configuration - For the patches that you want to remove, use the following
command:
oc label temporarypatch <patch_name> type-For example:
oc label temporarypatch wa-store-assistant-limits type-
- For all patches that you want to retain, use the following
command:
-
Run the following commands to verify that the temporary patches you want to preserve have the label
type=critical-configuration:oc get temporarypatch -n ${PROJECT_CPD_INST_OPERANDS} -l type=critical-configurationoc get temporarypatch -n ${PROJECT_CPD_INST_OPERANDS} --show-labels
Upgrading the service
cpd-cli
manage
apply-olm updates all of the OLM objects in the operators project
at the same time.To upgrade watsonx Assistant:
-
Log the
cpd-cliin to the Red Hat OpenShift Container Platform cluster:${CPDM_OC_LOGIN}Remember:CPDM_OC_LOGINis an alias for thecpd-cli manage login-to-ocpcommand. - Update the custom resource for watsonx Assistant.
cpd-cli manage apply-cr \ --components=watson_assistant \ --release=${VERSION} \ --cpd_instance_ns=${PROJECT_CPD_INST_OPERANDS} \ --param-file=/tmp/work/install-options.yml \ --license_acceptance=true \ --upgrade=true
Validating the upgrade
apply-cr command
returns:[SUCCESS]... The apply-cr command ran successfully
If you want to confirm that the custom resource status is
Completed, you can run the cpd-cli
manage
get-cr-status command:
cpd-cli manage get-cr-status \
--cpd_instance_ns=${PROJECT_CPD_INST_OPERANDS} \
--components=watson_assistant
Validating the EDB Postgres upgrade
- When do you need to complete this task?
- Complete this task after you successfully upgraded watsonx Assistant to Version 5.1.
- Set an instance environment variable.
export INSTANCE=wa - Make sure
wainstance goes to a fully verified state.oc get wa NAME VERSION READY READYREASON UPDATING UPDATINGREASON DEPLOYED VERIFIED QUIESCE AGE wa 5.1.0 True Stable False Stable 19/19 19/19 NOT_QUIESCED 25h - Check if
$INSTANCE-postgres-16cluster is in healthy or ready state.oc get cluster $INSTANCE-postgres-16 NAME AGE INSTANCES READY STATUS PRIMARY wa-postgres-16 22h 3 3 Cluster in healthy state wa-postgres-16-1oc get pods -l app=$INSTANCE-postgres-16 NAME READY STATUS RESTARTS AGE wa-postgres-16-1 1/1 Running 0 22h wa-postgres-16-2 1/1 Running 0 22h wa-postgres-16-3 1/1 Running 0 22h - Check whether all the jobs are in completed state.
oc get jobs -l slot=$INSTANCE NAME COMPLETIONS DURATION AGE wa-5-1-0-update-schema-store-db-job 1/1 2m22s 23h wa-cleanup-job 1/1 23h 23h wa-create-bucket-store-cos-job 1/1 12s 25h wa-create-slot-job 1/1 2m38s 25h wa-create-slot-store-db-job 1/1 42s 25h - Check whether the store cronjob is pointing to the new EDB Postgres-16
cluster.
oc get cronjob $INSTANCE-store-cronjob -o yaml | grep postgres # If today is the weekly backup day specified in postgres.backup.history.files.weeklyBackupDay then write the backup to the weekly dir value: wa-postgres-16-rw.cpd.svc name: wa-postgres-16-admin-auth name: wa-postgres-16-admin-auth - If
wa-postgres-16-rw.cpd.svcpoints to old EDB Postgres cluster instead of EDB Postgres-16, then delete the cronjob with the following command. This helps the operator to create a new cronjob with latest config.oc delete cronjob $INSTANCE-store-cronjob - Login to IBM Cloud Pak for Data console.
- Test the upgradation by sending a message to one of the old instances:
- Click Launch tool and click one of the watsonx Assistant service instances.
- Type a message in the preview chat window to trigger the actions that are defined for the watsonx Assistant before upgrade.
- Check whether you received the expected response without training the watsonx Assistant again.
Cleaning up resources
- Who needs to complete this task?
- You must be an administrator of the project where watsonx Assistant is installed.
- When do you need to complete this task?
- Complete this task to clean up resources after you upgrade watsonx Assistant to Version 5.1.
To delete the store-cronjob:
- Use the following command to delete the
store-cronjob:oc delete cronjob ${INSTANCE}-store-cronjob - Verify that the job is deleted
successfully:
After a few minutes, a new job with the same name must reappear.oc get cronjob -n ${PROJECT_CPD_INST_OPERANDS}
- Set an instance environment variable.
export INSTANCE=wa - Delete old EDB Postgres Version
12
cluster.
oc delete cluster -l app=$INSTANCE-postgres,component=postgres,service=conversation,slot=$INSTANCE - Delete old certificate created for EDB Postgres Version 12
cluster.
oc delete cert -l component=cert-postgres,service=conversation,slot=$INSTANCE - Delete old secrets created by the preceding
certificate.
oc delete secret -l component=cert-postgres,service=conversation,slot=$INSTANCE - Delete the watsonx Assistant created old
secrets.
oc delete secret -l app=$INSTANCE-postgres,component=postgres,service=conversation
Upgrading existing service instances
The service instances are automatically upgraded when you upgrade watsonx Assistant.
What to do next
watsonx Assistant is ready to use. To get started with watsonx Assistant, see Administering watsonx Assistant.