Guardium Data Security Center upgrade becomes stuck because CSV does not update
Symptoms
After upgrading Guardium Data Security Center Version 3.3.x (for example, from 3.3.3 to 3.3.4), the CSV does not update (for example, it does not update from CSV 3.3.3 to CSV 3.3.4).The Guardium Data Security Center operator log includes this error:
{"level":"info","ts":1715419159.915022,"logger":"logging_event_handler","msg":"[playbook task start]","name":"ispchz72s4pxldinqk76dfz8","namespace":"is","gvk":"gi.ds.isc.ibm.com/v1, Kind=TenantMinisnif","event_type":"playbook_on_task_start","job":"6302081700133434776","EventData.Name":"gi-global-defaults : Test if version is supported in general"}
--------------------------- Ansible Task StdOut -------------------------------
TASK [Test if version is supported in general] ********************************
fatal: [localhost]: FAILED! => {"changed": false, "msg": "Desired version: \"3.3.4\" is not supported. Operator supports Fresh Install at 3.3.3."}
-------------------------------------------------------------------------------
{"level":"error","ts":1715419160.0816762,"logger":"logging_event_handler","msg":"","name":"ispchz72s4pxldinqk76dfz8","namespace":"is","gvk":"gi.ds.isc.ibm.com/v1, Kind=TenantMinisnif","event_type":"runner_on_failed","job":"6302081700133434776","EventData.Task":"Test if version is supported in general","EventData.TaskArgs":"","EventData.FailedTaskPath":"/opt/ansible/roles/gi-global-defaults/tasks/main.yml:43","error":"[playbook task failed]","stacktrace":"github.com/operator-framework/operator-sdk/internal/ansible/events.loggingEventHandler.Handle\n\toperator-sdk/internal/ansible/events/log_events.go:111"}
----- Ansible Task Status Event StdOut (gi.ds.isc.ibm.com/v1, Kind=TenantMinisnif, ispchz72s4pxldinqk76dfz8/is) -----
PLAY RECAP *********************************************************************
localhost : ok=8 changed=0 unreachable=0 failed=1 skipped=7 rescued=0 ignored=0
Resolving the problem
- Determine the name of the job:
oc project openshift-marketplace oc get job -oyaml | less
In the output, locate
guardium-insights-operator-bundle
, get the job name, and delete it:oc delete job <name>
- Issue this command:
oc project openshift-operator-lifecycle-manager
and then restart all the pods in this project by running:oc delete $(oc get po -oname)
- Determine the name of the Guardium Data Security Center
installplan
:oc project sysqa oc get ip
And then delete the
installplan
:oc delete ip <Guardium Insights installplan name>
- Determine the name of the Guardium Data Security Center operator
deployment:
oc get deployment | grep guardium-insights-operator
And then delete the deployment:
oc delete deployment <Guardium Insights operator deployment name>
- Save the existing
subscription:
oc get sub ibm-guardium-insights-operator-subscription -oyaml > SaveLocation oc delete sub ibm-guardium-insights-operator-subscription vi SaveLocation
And then edit the subscription so that it looks like this:
apiVersion: operators.coreos.com/v1alpha1 kind: Subscription metadata: name: ibm-guardium-insights-operator-subscription spec: channel: v3.3 installPlanApproval: Automatic name: ibm-guardium-insights-operator source: ibm-guardium-insights-operator-catalog sourceNamespace: openshift-marketplace
And then apply the subscription:
oc apply SaveLocation