Secret watcher pod shows CrashLoopBackOff status
The secret-watcher
pod keeps crashing and shows a CrashLoopBackOff
status.
Cause
There might be many Kubernetes secrets that are configured in the cluster. The current memory limit that is configured in the secret-watcher
pod is not sufficient to manage the Kubernetes secrets. Kubernetes killed the pod due to less
memory.
Resolving the problem
To resolve the issue, increase the resource memory limit to a value that is sufficient for the pod to manage all the Kubernetes secrets.
-
Log in to your boot node with the
oc login
command. -
Check the number of Kubernetes secrets that are in the cluster.
oc get secrets --all-namespaces | wc -l
-
Edit the
CommonService
custom resource.oc -n <your-foundational-services-namespace> edit CommonService common-service
-
Add the following piece of configuration under the
spec.services
section. Add the required memory limit in theresources.limits.memory
section.spec: services: - name: ibm-im-operator spec: secretwatcher: resources: limits: memory: 220Mi
-
Save and exit the custom resource.
Wait for some time and then check the
secret-watcher
pod status. The status must show as1/1 Running
and the updated memory must show in the pod description. -
Get the
secret-watcher
pod status and name.oc -n <your-foundational-services-namespace> get pods | grep secret-watcher
-
Check the pod description.
oc -n <your-foundational-services-namespace> describe pod <secret-watcher-pod-name>