Upgrading the Red Hat OpenShift cluster that is hosting IBM Concert Operate
When planned correctly and completed within supported version ranges, upgrades of Red Hat OpenShift Container Platform usually complete without disruption to IBM Concert Operate.
Overview
IBM Concert Operate has in-cluster high-availability, and uses PodDisruptionBudgets (PDBs) that are designed to allow Red Hat OpenShift upgrades to proceed without manual intervention. In typical production deployments, if Red Hat OpenShift reports that a node cannot be drained due to a PDB, this is usually because more processing and reconciliation time is needed or because of a problem with application health. Starter non-HA deployments of IBM Concert Operate have components that might use PDBs that permit zero disruptions, and manual intervention might be required to allow upgrades to progress.
A Red Hat OpenShift cluster that is running IBM Concert Operate takes longer to upgrade than an empty Red Hat OpenShift cluster due to the additional work required to evict and reschedule active workloads. This behavior is expected.
Pre-upgrade procedure
Before you start your Red Hat OpenShift upgrade, use the following steps to help ensure that the process completes with minimal to no downtime for IBM Concert Operate.
1. Check version compatibility
2. Check adequate compute resource headroom
3. Check adequate storage headroom
When you are ready to upgrade Red Hat OpenShift, use the instructions in the [Red Hat OpenShift documentation .
1. Check version compatibility
Check that your current IBM Concert Operate version is compatible with your target Red Hat OpenShift version.
Each Red Hat OpenShift release includes updates to the underlying Kubernetes API and Red Hat OpenShift services. To avoid compatibility issues and problems, your currently installed version of IBM Concert Operate must support the version of Red Hat OpenShift that you are upgrading to. For more information about the versions of Red Hat OpenShift that are supported by IBM Concert Operate, see Supported platforms: Red Hat OpenShift Container Platform.
If your currently installed version of IBM Concert Operate does not support the version of Red Hat OpenShift that you are upgrading to, then upgrade IBM Concert Operate to a version that does. You must upgrade IBM Concert Operate before you upgrade Red Hat OpenShift. For more information about upgrading Red Hat OpenShift and IBM Concert Operate, see Guidance for upgrades that require an Red Hat OpenShift upgrade.
2. Check adequate compute resource headroom
As part of the Red Hat OpenShift upgrade, the control plane nodes are updated. IBM Concert Operate operators use the Kubernetes API services that are hosted on the control plane nodes to manage the application workloads. If a problem occurs during this update, it might affect IBM Concert Operate's lifecycle management, but this is not expected.
- Cordoned (no new pods scheduled)
- Drained (running pods are evicted)
- Upgraded and restarted
- Returned to service
You must adhere to the cluster sizing guidance to help ensure that adequate cluster resources are available for workloads to run on without interruption while nodes are updated. For more information about sizing and hardware requirements, see Hardware requirements.
3. Check adequate storage headroom
The storage layer often needs to be upgraded alongside Red Hat OpenShift to maintain compliance with its own Red Hat OpenShift version compatibility requirements - especially if you are using Red Hat OpenShift Data Foundation.
If a problem occurs in the storage layer, IBM Concert Operate is impacted because of its dependency on the storage.
When each node that hosts storage is drained and reloaded during the Red Hat OpenShift upgrade, IBM Concert Operate temporarily runs in a degraded state because it does not have its full complement of storage nodes. If the storage is already at or near capacity, the node drainage can push the storage layer into a critical state where writes to it are disabled and IBM Concert Operate is impacted. To help avoid this occurring, ensure that adequate storage is available for the upgrade - ideally a storage utilization of 60% or less, and definitely less than 74%.
For more information about sizing and hardware requirements, see Storage requirements.
4. Verify application health
Verify the health of IBM Concert Operate components and confirm that no workloads are already degraded.
mustgather healthcheck tool to verify that your IBM Concert Operate deployment is in a healthy state before you commence your Red Hat OpenShift upgrade. For more information, see the following topics: