IBM Cloud Pak foundational services upgrade from 3.6.6 to 3.19.8 fails

Firstly, you need to check whether the ODLM installs plan has been generated by checking the ODLM in the Red Hat OpenShift UI. If it hasn't been generated, it means that the OLM is stuck and unable to generate an install plan for ODLM.

In the catalog-operator logs, there is a complaint about the existence of subscriptions for catalog-ui and monitoring-grafana. To resolve this, you can remove the subscription for ibm-catalog-ui-operator in the Ibm-common-services namespace and check whether ODLM is installed in the cluster.

If the ODLM install plan is still not generated, you should inspect the log of the olm-operator. It can be complaining about the Helm-api and helm-repo operators. In this case, you can remove the subscription and csv of those operators.

Finally, you need to manually restart the ibm-common-service-operator pod to regenerate the subscription of ODLM.

After generating the install plan, the upgrade of ODLM and other operators should be successful. However, if the pods of the monitoring-grafana operator are still running, you need to check the deployment. It's possible that the operand of that operator wasn't removed. In this case, you should delete the operand to ensure that all other pods are functioning properly and that the certificates remain valid.