Cleaning up a failed management upgrade
If the management upgrade fails, remove any unneeded artifacts.
About this task
You can remove artifacts left from the failed upgrade, and then either retry the upgrade or stop the upgrade process.
Procedure
- Determine if you have any v10 ingresses left after successful rollback, and delete
them:
- Check for
ingresses:
kubectl get ingress -n <management-subsystem-namespace>
If any of the following ingresses are found, delete them manually:
<management-subsystem-name>-admin
<management-subsystem-name>-api-manager
<management-subsystem-name>-consumer-api
<management-subsystem-name>-platform-api
Note: v10 ingresses are not deleted if the upgrade operator enteredFailed
status during any of these upgrade steps:- Creating postgres cluster
- Running load job
- Bringing up v10 deployments
- To manually delete the v10 ingresses after a
rollback:
kubectl delete ingress <ingress-name> -n <management-subsystem-namespace>
Note: Delete only the v10 ingresses (if any). If not sure, compare the ingresses saved before performing the management upgrade to v10.
- Check for
ingresses:
- Use the following instructions to manually delete the PVC used by the upgrade
process: Note: Why is this step necessary?
The PVC used by extract and load job has the same name as that of the management upgrade CR name. Rolling back the management upgrade process does not delete the PVC used by extract/load jobs. This is done to ensure that you can access to the extract and load jobs after a rollback or a successful upgrade.
- If applicable, gather extract and load job logs. See Logs for extract and load jobs .
- Gather logs for the upgrade operator. See Logs for management upgrade.
- Manually delete the PVC:Important: Save the PVC data before deleting it.
- Delete extract
jobs:
kubectl delete jobs v2018-to-v10-extract -n <management-subsystem-namespace>
- Delete load
jobs:
kubectl delete jobs v2018-to-v10-load -n <management-subsystem-namespace>
- Get the PVC name. The PVC name is the same as the name of the management upgrade
CR.
kubectl get pvc -n <management-subsystem-namespace>
- Delete the
PVC:
kubectl delete pvc <management-upgrade-CR-name> -n <management-subsystem-namespace>
- Delete extract
jobs:
- If necessary, delete the postgres PVCs.
The postgres PVCs must be deleted if you triggered a rollback on or after the upgrade reconciles the v10 cluster with only a postgres database.
- List the postgres
PVCs:
kubectl get pvc -n <management-subsystem-namespace>
The following PVCs are used by
postgres
and must be deleted manually:mgmt-xxxxxxxx-postgres
mgmt-xxxxxxxx-postgres-pgbr-repo
mgmt-xxxxxxxx-postgres-wal
- Delete the
PVCs:
kubectl delete pvc mgmt-xxxxxxxx-postgres -n <management-subsystem-namespace> kubectl delete pvc mgmt-xxxxxxxx-postgres-pgbr-repo -n <management-subsystem-namespace> kubectl delete pvc mgmt-xxxxxxxx-postgres-wal -n <management-subsystem-namespace>
- List the postgres
PVCs:
- Decide whether to troubleshoot and retry the management upgrade to v10, or to stop the
upgrade process.
- To troubleshoot and retry, continue with Step 5.
- To stop the upgrade process, go to Stopping the v2018 to v10 management upgrade.
- To re-run the management upgrade from v2018 to v10:
- Delete management upgrade
CR:
kubectl delete up2018m <management-subsystem-name> -n <management-subsystem-namespace>
- Fix the errors in your v2018 installation. See Common errors on management upgrades.
- Re-create the management upgrade v10 CR to trigger v10 management upgrade. See Preparing the 2018 deployment for upgrade.
- Delete management upgrade
CR: