Ingress Nginx operator failed to create operand due to resource exists

When you install IBM Cloud Pak foundational services, the Ingress Nginx operator fails to create the operand.

Symptoms

The Ingress Nginx operand was not created. From the Ingress Nginx operator, you see log entries similar to this example.

{"level":"error","ts":1600391663.8330877,"logger":"controller","msg":"Reconciler error","controller":"nginxingress-controller","name":"default","namespace":"ibm-common-services","error":"failed to install release: rendered manifests contain a resource that already exists. Unable to continue with install: ClusterRole \"nginx-ingress-clusterrole\" in namespace \"\" exists and cannot be imported into the current release: invalid ownership metadata; label validation error: missing key \"app.kubernetes.io/managed-by\": must be set to \"Helm\"; annotation validation error: missing key \"meta.helm.sh/release-name\": must be set to \"default\"; annotation validation error: missing key \"meta.helm.sh/release-namespace\": must be set to \"ibm-common-services\"",

Cause

Two possible causes can lead to this problem.

1. The cluster is not clean and still contains resources from a previous installation.
2. The Helm API operator installation failed which didn't clean up all the resources. The second attempt to install failed due to resource exists.

Resolution

Delete the resources that were already created.

# oc get clusterrole | grep nginx-ingress
nginx-ingress-clusterrole                                              2d14h

# oc get clusterrolebinding | grep nginx-ingress
nginx-ingress-clusterrole-nisa-binding                                            2d14h

 oc get scc | grep nginx-ingress
nginx-ingress-scc        2d14h

# oc delete clusterrole nginx-ingress-clusterrole
clusterrole.rbac.authorization.k8s.io "nginx-ingress-clusterrole" deleted

# oc delete clusterrolebinding nginx-ingress-clusterrole-nisa-binding
clusterrolebinding.rbac.authorization.k8s.io "nginx-ingress-clusterrole-nisa-binding" deleted

# oc delete scc nginx-ingress-scc
securitycontextconstraints.security.openshift.io "nginx-ingress-scc" deleted

Restart the ibm-ingress-nginx-operator pod to create operand resources.

# oc delete pods ibm-ingress-nginx-operator-57495f9f6d-r6j8h
pod "ibm-ingress-nginx-operator-57495f9f6d-r6j8h" deleted

After a few minutes, you can see the pod and Ingress NGINX operand.

# oc get pod | grep nginx
ibm-ingress-nginx-operator-57495f9f6d-sqgt9              1/1     Running            0          12m
nginx-ingress-controller-8677bdb8-59czb                  1/1     Running            0          12m
nginx-ingress-controller-8677bdb8-lg4hz                  1/1     Running            0          12m

# oc get configmap ibmcloud-cluster-info -o yaml
apiVersion: v1
data:
  cluster_address: cp-console.apps.debenhams.os.fyre.ibm.com
  cluster_ca_domain: cp-console.apps.debenhams.os.fyre.ibm.com
  cluster_endpoint: https://icp-management-ingress.ibm-common-services.svc:443
  cluster_kube_apiserver_host: api.debenhams.os.fyre.ibm.com
  cluster_kube_apiserver_port: "6443"
  cluster_name: mycluster
  cluster_router_http_port: "80"
  cluster_router_https_port: "443"
  openshift_router_base_domain: apps.debenhams.os.fyre.ibm.com
  proxy_address: cp-proxy.apps.debenhams.os.fyre.ibm.com
  proxy_ingress_http_port: "80"
  proxy_ingress_https_port: "443"
  version: 3.5.0
kind: ConfigMap