Preparing to upgrade Watson Machine Learning

If you plan to upgrade Watson Machine Learning on IBM® Cloud Pak for Data, a cluster administrator must update the cluster environment for Watson Machine Learning.

Before you begin

Required role: To complete this task, you must be a cluster administrator.

Before you update the cluster for Watson Machine Learning, ensure that:

The Cloud Pak for Data control plane is already upgraded on your Red Hat® OpenShift® cluster. For details, see Upgrading IBM Cloud Pak for Data.
The cluster meets the minimum requirements for Watson Machine Learning. For details, see System requirements for services.
The machine from which you will run the commands meets the requirements described in Preparing your installation node.
On air-gapped clusters: You completed the steps in Preparing for air-gapped upgrades to download the required files for the service.

Tip: For a list of all available options, enter the following command:

./cpd-cli adm --help

Training job pods

Before upgrading Cloud Pak for Data, perform the following check to determine if any training job pods are still running:

oc get pods -l app=training-job

Next, check if any training job computation pods are running:

oc get pods -l app=training-computations

If no such pods are running, proceed with the upgrade. If pods are still running, wait for the training jobs to complete before beginning the upgrade.

Note: If the training does not complete within an hour, contact IBM customer support to investigate why the job is not completing.

Deployment batch job pods

Before upgrading Cloud Pak for Data, perform the following check to determine if any batch job pods are still running:

oc get pods | grep "wml-dep-bd"

Next, check that all of the scheduled batch jobs are not falling in the upgrade time window.

Verify the PVC access mode

If you are upgrading Cloud Pak for Data from version 3.0.1 to version 3.5, verify the PVC access mode for manager-pvc and etcd-pvc on the existing Cloud Pak for Data 3.0.1 cluster. Type the following command:

oc get pvc | egrep "etcd-pvc|manager-pvc"

Example output:

etcd-pvc                                      Bound    pvc-a68bd96f-c438-4700-9b79-490cecad8995   10Gi       RWO            portworx-gp3-sc           
manager-pvc                                   Bound    pvc-bfaa22ce-0246-421e-9272-2194f9afabe4   10Gi       RWX            portworx-shared-gp3

For Portworx, if etcd-pvc has RWO access mode and manager-pvc has RWX access mode, use the default Portworx override file portworx-override-x86.yaml during upgrade.

For PPC with NFS, if manager-pvc has RWX access mode, use the default nfs-override-ppc64le.yaml file during upgrade.

For Red Hat OpenShift Container Storage, if etcd-pvc has RWO access mode and manager-pvc has RWX access mode, use the default Portworx override file ocs-override-x86.yaml during upgrade.

Note:

If the access mode permissions are different for any of etcd-pvc or manager-pvc, do not proceed with the upgrade. Contact IBM technical support team for information on what steps to follow.

Procedure

Prepare the cluster for Watson Machine Learning by completing the appropriate task for your environment:
- Preparing clusters connected to the internet
- Preparing air-gapped clusters
Complete the tasks listed in What to do next

Preparing clusters connected to the internet

From your installation node:

Change to the directory where you placed the Cloud Pak for Data command-line interface and the repo.yaml file.
Log in to your Red Hat OpenShift cluster as an administrator:
```
oc login OpenShift_URL:port
```

Run the following command to see a preview of the list of resources that must be created or updated on the cluster:

./cpd-cli adm \
--repo ./repo.yaml \
--assembly wml \
--arch Cluster_architecture \  
--namespace Project \
--latest-dependency

Important: By default, this command gets the latest version of the assembly. If you want to install a specific version of Watson Machine Learning, add the following line to your command after the --assembly flag:

--version Assembly_version \

The --latest-dependency flag gets the latest version of the dependent assemblies. If you remove the --latest-dependency flag, the installer will either leave the dependent assemblies at their current version or will update the dependent assemblies to the minimum required version.

Tell the person who will upgrade Watson Machine Learning whether you used either of these flags. The upgrade command must be run with the same flags.

Replace the following values:

Variable	Replace with
`Assembly_version`	The version of Watson Machine Learning that you want to install. The assembly versions are listed in System requirements for services.
`Cluster_architecture`	Specify the architecture of your cluster hardware: For x86-64 hardware, remove this flag or specify `x86_64`
`Project`	The project where the Cloud Pak for Data control plane is installed.

The command returns a list of the changes that you must make to your cluster to ensure that Watson Machine Learning can run on your cluster.

Make the necessary changes to your cluster. You can choose one of the following methods to make the changes:
To automatically apply the changes to your cluster:
Re-run the cpd adm command with the --apply flag:
```
./cpd-cli adm \
--repo repo.yaml \
--assembly wml \
--arch Cluster_architecture \
--namespace Project \
--latest-dependency \
--apply
```
Replace the variables with the same values that you used the last time you ran the command.
To manually apply the changes to your cluster:

Follow the appropriate procedures from the Red Hat OpenShift documentation to complete the required tasks.

Preparing air-gapped clusters

From your installation node:

Change to the directory where you placed the Cloud Pak for Data command-line interface.
Log in to your Red Hat OpenShift cluster as an administrator:
```
oc login OpenShift_URL:port
```

Run the following command to see a preview the list of resources that must be created or updated on the cluster:

./cpd-cli adm \
--assembly wml \
--arch Cluster_architecture \
--namespace Project \
--load-from Image_directory_location \
--latest-dependency

Note: If the assembly was downloaded using the delta-images command, remove the --latest-dependency flag from the command. If you don't remove the --latest-dependency flag you will get an error indicating that the flag cannot be used.

Tell the person who will upgrade Watson Machine Learning whether you used the --latest-dependency flag. If you run this command with the --latest-dependency flag, the upgrade command must also be run with the flag.

Replace the following values:

Variable	Replace with
`Cluster_architecture`	Specify the architecture of your cluster hardware: For x86-64 hardware, remove this flag or specify `x86_64`
`Project`	The project where the Cloud Pak for Data control plane is installed.
`Image_directory_location`	The location of the cpd-cli-workspace directory.

The command returns a list of the changes that you must make to your cluster to ensure that Watson Machine Learning can run on your cluster.

Make the necessary changes to your cluster. You can choose one of the following methods to make the changes:
To automatically apply the changes to your cluster:
Re-run the cpd adm command with the --apply flag:
```
./cpd-cli adm \
--assembly wml \
--arch Cluster_architecture \
--namespace Project \
--load-from Image_directory_location \
--latest-dependency \
--apply
```
Replace the variables with the same values that you used the last time you ran the command.
To manually apply the changes to your cluster:

Follow the appropriate procedures from the Red Hat OpenShift documentation to complete the required tasks.

What to do next

The service is ready to be upgraded. For details, see Upgrading Watson™ Machine Learning.