IBM Support

ODF Operator cannot upgrade due to `a unique replacement chain within a channel is required`

Troubleshooting


Problem

ODF operator cannot upgrade due to error:
 
Message: a unique replacement chain within a channel is required to determine the relative order between channel entries, but 2 replacement chains were found in channel "" of package "ocs-operator"

Cause

This problem is an OLM-related issue. Further information can be found in the following Red Hat Knowledgebase articles:

Environment

Red Hat OpenShift Data Foundation 4.8 - 4.10 deployed on:

  • IBM Cloud ROKS
  • IBM Cloud Satellite

Diagnosing The Problem

Check the csv output. For example:

$ oc get csv |grep "ocs-operator"
NAME                                     DISPLAY                       VERSION   REPLACES                            PHASE
ocs-operator.v4.10.11                    OpenShift Container Storage   4.10.11   ocs-operator.v4.10.10               Replacing
ocs-operator.v4.10.14                    OpenShift Container Storage   4.10.14   ocs-operator.v4.10.11               Pending

 

The subscription to the operator contains the following message: 

# For example from "oc describe subscription -n openshift-storage" command output:

Message:                a unique replacement chain within a channel is required to determine the relative order between channel entries, but 2 replacement chains were found in channel "" of package "ocs-operator": ocs-operator.v4.10.14, ocs-operator.v4.10.11
    Reason:                 ErrorPreventedResolution
    Status:                 True
    Type:                   ResolutionFailed

 

Check the status of the pod and deployment of the operator if they're up and running:

$ oc get all -owide -n openshift-storage
 

Resolving The Problem

In the openshift-storage namespace scale down the deployment for the ocs-operator and rook-ceph-operator, & validate the deployment goes 0/0 and wait for those pods to successfully terminate:

$ oc scale deployment ocs-operator rook-ceph-operator --replicas=0 -n openshift-storage
$ oc get deployment -n openshift-storage
$ oc get pods -n openshift-storage
 
  • Wait a minimum of 5 minutes with the operators scaled down.
 

Then restart the catalog-operator and olm-operator pods in the openshift-operator-lifecycle-manager namespace, and validate the new pods are running:

$ oc delete pod -l app=catalog-operator -n openshift-operator-lifecycle-manager
$ oc delete pod -l app=olm-operator -n openshift-operator-lifecycle-manager
$ oc get pods -n openshift-operator-lifecycle-manager
 

After these olm pods come back up, scale back up the operators and wait for the deployments to go 1/1 and for the pods to get created:

$ oc scale deployment ocs-operator rook-ceph-operator --replicas=1 -n openshift-storage
$ oc get deployment -n openshift-storage
$ oc get pods -n openshift-storage
 

This should allow for OpenShifts OLM to pull in the data fresh and kick off the upgrade. If the operation is successful you will start to see the ODF pods restart, and as the upgrade progresses the versions get updated in the ClusterServiceVersion and can be monitored with the following:

$ watch `oc get pods -n openshift-storage`
$ oc get csv -n openshift-storage
 
 

If the steps above don't work, perform the following steps:

1. Take the backup of the CatalogSource, subscription, and InstallPlans.

2. Disable the CatalogSource by using the following command:

$ oc patch OperatorHub cluster --type json -p '[{"op": "add", "path": "/spec/sources", "value": [{"disabled": true, "name": "redhat-operators"}]}]'

3. Create the new CatalogSource by pointing to the next ODF version that you're upgrading to:

# For example if you're upgrading to 4.11, mention it like the following:

$ cat <<EOF | oc apply -f -
apiVersion: operators.coreos.com/v1alpha1
kind: CatalogSource
metadata:
  name: redhat-operators
  namespace: openshift-marketplace
spec:
  displayName: Openshift Container Storage
  icon:
    base64data: ""
    mediatype: ""
  image: registry.redhat.io/redhat/redhat-operator-index:v4.11
  publisher: Red Hat
  sourceType: grpc
  priority: 100
EOF

4. Check that the CatalogSource is created successfully by checking, redhat-operators-xxx pod:

$ oc get po -n openshift-marketplace

5. Now edit the subscription and change the channel to the next version that CatalogSource pointing to in the UI and try upgrading again.

6. Check whether the upgrade started successfully and is progressing:

$ oc get ip -n openshift-storage
$ oc get subs -n openshift-storage
$ oc get csv -n openshift-storage
$ oc get storagecluster -n openshift-storage
$ oc get pods -n openshift-storage

7. After the current upgrade is completed successfully, delete the CatalogSource that is created manually and enable the CatalogSource further:

$ oc delete CatalogSource redhat-operators -n openshift-marketplace
$ oc patch OperatorHub cluster --type json -p '[{"op": "add", "path": "/spec/sources", "value": [{"disabled": false, "name": "redhat-operators"}]}]'

Document Location

Worldwide

[{"Type":"MASTER","Line of Business":{"code":"LOB66","label":"Technology Lifecycle Services"},"Business Unit":{"code":"BU070","label":"IBM Infrastructure"},"Product":{"code":"SSSEWFV","label":"Storage Fusion Data Foundation"},"ARM Category":[{"code":"a8m3p000000UoIPAA0","label":"Support Reference Guide"}],"ARM Case Number":"TS013856650","Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"All Versions"}]

Document Information

Modified date:
11 November 2025

UID

ibm17029642

Page Feedback