Reducing system downtime

Configure a database state to reduce Maximo® Manage system downtime when you upgrade to Maximo Manage 9.0 and later.

Before you begin

onlineUpgrade is supported on Db2® only from Maximo Manage 8.6.2. Support for other databases is provided only after Maximo Manage 9.0.0. The operator displays an error if you attempt to run an online or offline upgrade, and checks if maxupg indicates a version older than 8.6.2.

About this task

The ManageWorkspace custom resource (CR) has the spec.settings.db.upgrade.upgradeType object, which has enum property values that correspond to the following states for reduced system downtime:

enum value	Description
`regularUpgrade`	The regular type of upgrade where the server is down during the entire database upgrade.
`onlineUpgrade`	Two-phased upgrade with offline phase triggered. The server is down only during the offline phase of the database upgrade.

onlineUpgrade reduces the system downtime period and provides better control when the system is down.

Note:

The upgrade setting affects how the database is updated during version update, as well as when a new add-on is added to Maximo Manage.
Before you add an industry solution, set the upgradeType value to regularUpgrade. After the industry solution is installed, set the upgradeType to onlineUpgrade for future updates.

Procedure

Log in to Red Hat® OpenShift® Container Platform.
From the side navigation menu, click Administration > CustomResourceDefinitions.
On the CustomResourceDefinitions page, search for ManageWorkspace.
Click ManageWorkspace and then click the Instances tab.
Select an instance and click the YAML tab.
In the settings.upgrade.failureControl section, set a value to control the upgrade failure:

retry

The default value for all reduce downtime states.

Retries the upgrade if the target status is not reached for a particular stage of the upgrade.

rollbackForOnlineUpgrade

Once it is set, and the upgradeType is not regular upgrade, the operator attempts to roll back the change.

Once rolled back, the operator does not consider it as a reconcile failure.

It reflects the result in the status, but not in retry.

Use the settings.upgrade.failureControl section to control the operator behavior if there are upgrade failures. Set it to rollbackForOnlineUpgrade to roll back the online portion of the upgrade or to stop the retrying. It gives you the time to fix the database.

Note: The roll back can be done only for the online portion of the database change. If a failure happens during the offline portion of the upgrade, the process can continue only when the database is fixed, or manually recovered by using a backup of the original state, or at the state when the online upgrade is completed.
If you want to run the version of Maximo Manage that you had before the upgrade, here are the following options:
- Starting in Maximo Application Suite 9.1, you can remain in the current state but it has two limitations:
  - If there is a serious issue with the cluster that results in the loss of the ManageServerBundle custom resources, the system does not revert as-is.
  - You cannot change the system apart from the podTemplates and the replica size of the bundle servers.
- Roll back the Maximo Manage operator. Set the ManageApp custom resource version field to the previous Maximo Manage version.
- Retain the already upgraded operator, and set the ManageWorkspace custom resource's base and other components to the earlier version.
- Retain the already upgraded operator and set the ManageWorkspace custom resource'sspec.settings.deployment.buildTag to the previous build tag to match the version of Maximo Manage before the upgrade. You can find the build tag from ImageStream. Make a note of the build tag and make sure that it is not purged before you start the upgrade. When you are ready to do the upgrade after you fix the database, set the build tag to latest.
Starting in Maximo Application Suite 9.1, on failure of the online upgrade stage, if roll back is set for failureControl:
- The operator shuts down the servers, rolls back the database, and restarts the servers.
- The server bundles run with the older version image after the roll back.
Optional: To trigger the offline portion of the upgrade, use toolsAPI or directly use the Maximo Manage admin console to issue the start-offline-upgrade.sh command. Alternatively, directly update the ManageOfflineUpgrade custom resource's value from waiting to requested.
When onlineUpgrade is used, after the online portion of the upgrade is completed, the operator creates a waiting value in the ManageOfflineUpgrade custom resource in the Maximo Manage namespace.
```
spec
 stage: waiting
status:
```
Starting in Maximo Application Suite 9.1, after the online upgrade is started, and before the offline portion is finished, the changes to the deployment of the server bundles is limited. Scaling the server bundles by changing the replica size is possible only through podTemplates.

Results

The stage of the upgrade and any upgrade failures are reported in the DBReady condition of the ManageWorkspacecustom resource.

In the MAXVAR table, the SEAMLESSUPGRADE value indicates the status of the database during, before, and after the upgrade:

Condition	Value
When the system is started from 862	The value is left to be 0.
When the online portion of the upgrade starts	The value is set to 1.
When the online portion of the upgrade is completed	The value is set to 2. If there is an error in the online upgrade, the value remains at 1.
When the offline portion of the upgrade starts	The value is set to 3.
When the offline portion of the upgrade is completed	The value is set to 0.
When the roll back starts	The value is set to 4. It can be rolled back only if the MAXVAR is set to 1. If the value is 2, you can only go forward or restore the database.
When the roll back is completed	The value is set to 0.

Note:

0: The database is consistent, there is no partial upgrade.
1: The database started the online phase of the upgrade, but it did not complete or failed.
2: The database online upgrade phase finished completed and stayed at this stage.
3: The offline portion of the upgrade started but it did not finish or failed.
4: The roll back or the online portion of the upgrade is in progress.

The upgrade stops and operator reports a failure if the database is of SEAMLESSUPGRADE=1 or SEAMLESSUPGRADE=2, indicating a failed or partially complete upgrade, while the MASIMAGEIDSTART recorded by the database does not match the current image ID. It is an indication that the image might mismatch. Therefore, the operator cannot ensure that it can roll back or upgrade the database, and reports an error.