Fusion Data Foundation service install and upgrade issues

Use this troubleshooting information to resolve install and upgrade problems that are related to Fusion Data Foundation service.

Known issues in upgrade

  • You cannot upgrade Fusion Data Foundation from 4.14 or 4.15 to 4.16 instead do a fresh installation of Fusion Data Foundation 4.16.

ceph-csi-controller-manager pod in Crashloopbackoff state

Problem statement

After the upgrade of OpenShift® Container Platform 4.16 and Fusion Data Foundation 4.16 to 4.17, the Fusion Data Foundation goes into a degraded state.

Cause
It error occurs because of the Crashloopbackoff state of the ceph-csi-controller-manager pod.
Resolution
Do the following steps to resolve and validate this error:
  1. Run the following example command to remove the limit from CSV: 
    oc patch csv cephcsi-operator.v4.17.1 --type json -p '[ {"op": "remove", "path": "/spec/install/spec/deployments/0/spec/template/spec/containers/1/resources/limits"} ]' 
clusterserviceversion.operators.coreos.com/cephcsi-operator.v4.17.1 patched
  2. Run the following command to verify the pods status: 
    oc get pods
    Example output: 
    NAME                                                              READY   STATUS    RESTARTS      AGE
ceph-csi-controller-manager-675dc67f7b-f64pb                      2/2     Running   0             3h39m
ceph-csi-controller-manager-8854c7f5-xdrvx                        1/2     Running   0             6s

Data Foundation operator upgrade is not initiated

Problem statement
Data Foundation operator upgrade is not initiated even though the service is installed successfully. If auto-upgrade is not enabled for Data Foundation during the upgrade to version 4.17 or later, some components might not upgrade automatically.
Resolution
To ensure all components are upgraded successfully, verify that all InstallPlans in the openshift-storage namespace are approved. Follow the steps to verify and manually approve any pending InstallPlans:
  1. Run the following command to retrieve the list of InstallPlans in the openshift-storage namespace.
    oc get installplan -n openshift-storage
  2. Check the APPROVED column in the output and make sure that all listed InstallPlans status set to true.
  3. If any InstallPlans in the APPROVED column is set to false, then manually approve it by running the following command.
    for ip in $(oc get installplan --no-headers -n openshift-storage | grep false | awk '{print $1}')
do
  oc patch installplan $ip -n openshift-storage --type merge --patch '{"spec":{"approved":true}}'
done
  4. After approving all pending InstallPlans, repeat the steps 1 and 2 to confirm that all InstallPlans status set to true.

Data Foundation ocs-client-operator-console-xxx pod is stuck while upgrade

Problem statement
During the Data Foundation service upgrade from version 4.18.2 to 4.18.3, an error occurred: the odf-operator CSV ocs-operator.v4.18.3 is unavailable due to ocs-client-operator-console-xxx pod is stuck in ContainerCreating state.
Resolution
Delete the pod to resolve the issue using the following command.
oc delete pod <name> -nopenshift-storage