Troubleshooting and deleting remaining resources during Uninstall

Occasionally some of the custom resources managed by an operator may remain in Terminating status waiting on the finalizer to complete, although you have performed all the required cleanup tasks. In such an event you need to force the removal of such resources. If you do not do so, the resources remain in the Terminating state even after you have performed all the uninstall steps.

1. Check if the openshift-storage namespace is stuck in Terminating state upon deletion.

   oc get project -n <namespace>

   Output:

   NAME DISPLAY NAME STATUS
   openshift-storage Terminating

2. Check for the NamespaceFinalizersRemaining and NamespaceContentRemaining messages in the STATUS section of the command output and perform the next step for each of the listed resources.

   oc get project openshift-storage -o yaml

   Example output :

   status:
     conditions:
     - lastTransitionTime: "2020-07-26T12:32:56Z"
       message: All resources successfully discovered
       reason: ResourcesDiscovered
       status: "False"
       type: NamespaceDeletionDiscoveryFailure
     - lastTransitionTime: "2020-07-26T12:32:56Z"
       message: All legacy kube types successfully parsed
       reason: ParsedGroupVersions
       status: "False"
       type: NamespaceDeletionGroupVersionParsingFailure
     - lastTransitionTime: "2020-07-26T12:32:56Z"
       message: All content successfully deleted, may be waiting on finalization
       reason: ContentDeleted
       status: "False"
       type: NamespaceDeletionContentFailure
     - lastTransitionTime: "2020-07-26T12:32:56Z"
       message: 'Some resources are remaining: cephobjectstoreusers.ceph.rook.io has
         1 resource instances'
       reason: SomeResourcesRemain
       status: "True"
       type: NamespaceContentRemaining
     - lastTransitionTime: "2020-07-26T12:32:56Z"
       message: 'Some content in the namespace has finalizers remaining: cephobjectstoreuser.ceph.rook.io
         in 1 resource instances'
       reason: SomeFinalizersRemain
       status: "True"
       type: NamespaceFinalizersRemaining

3. Delete all the remaining resources listed in the previous step.

   For each of the resources to be deleted, perform the following:

   1. Get the object kind of the resource which needs to be removed.
      See the message in the above output.

      Example, where cephobjectstoreuser.ceph.rook.io is the object kind.

      message: Some content in the namespace has finalizers remaining:
      cephobjectstoreuser.ceph.rook.io

   2. Get the Object name corresponding to the object kind.

      oc get <Object-kind> -n <project-name>

      Example:

      oc get cephobjectstoreusers.ceph.rook.io -n openshift-storage

      Example output:

      NAME                           AGE
      noobaa-ceph-objectstore-user   26h

   3. Patch the resources.

      oc patch -n <project-name> <object-kind>/<object-name> --type=merge -p '{"metadata": {"finalizers":null}}'

      Example :

      oc patch -n openshift-storage cephobjectstoreusers.ceph.rook.io/noobaa-ceph-objectstore-user \
      --type=merge -p '{"metadata": {"finalizers":null}}'

      Example output:

      cephobjectstoreuser.ceph.rook.io/noobaa-ceph-objectstore-user patched

4. Verify that the openshift-storage project is deleted.

   oc get project openshift-storage

   Example output:

   Error from server (NotFound): namespaces "openshift-storage" not found