auth-idp pod restarts multiple times
The auth-idp pod restarts multiple times.
Symptoms
The auth-idp pod restarts multiple times. The platform-auth-service shows the following logs:
kubectl logs <auth-idp pod name> -n kube-system -c platform-auth-service
2019-02-12 18:18:12,170 WARN received SIGTERM indicating exit request
2019-02-12 18:18:12,170 INFO waiting for 01-wlp-platform-auth-service, 02-directory-service to die
2019-02-12 18:18:12,183 INFO stopped: 02-directory-service (terminated by SIGTERM)
[AUDIT ] CWWKE0085I: The server defaultServer is stopping because the JVM is exiting.
[AUDIT ] CWWKE1100I: Waiting for up to 30 seconds for the server to quiesce.
[AUDIT ] CWWKT0017I: Web application removed (default_host): http://auth-idp-v9h8q:9080/oidc/
[AUDIT ] CWWKT0017I: Web application removed (default_host): http://auth-idp-v9h8q:9080/oauth2/
2019-02-12 18:18:15,385 INFO waiting for 01-wlp-platform-auth-service to die
2019-02-12 18:18:18,387 INFO waiting for 01-wlp-platform-auth-service to die
2019-02-12 18:18:21,390 INFO waiting for 01-wlp-platform-auth-service to die
2019-02-12 18:18:22,391 WARN killing '01-wlp-platform-auth-service' (9) with SIGKILL
2019-02-12 18:18:22,514 INFO stopped: 01-wlp-platform-auth-service (terminated by SIGKILL)
Causes
The platform-auth-service is using more resources than the configured limit. This issue is seen more often on Linux on IBM Z and LinuxONE platforms.
Resolving the problem
To resolve the issue, remove the resource limits.
-
Install kubectl. See Installing the Kubernetes CLI (kubectl).
-
Edit the
auth-idpdaemonset.kubectl edit ds auth-idp -n kube-system -
Locate the following section:
name: platform-auth-service ports: - containerPort: 8443 hostPort: 8443 name: http protocol: TCP readinessProbe: failureThreshold: 3 httpGet: path: / port: 8443 scheme: HTTPS periodSeconds: 10 successThreshold: 1 timeoutSeconds: 1 resources: limits: cpu: "1" memory: 1Gi requests: cpu: 100m memory: 256Mi -
Remove the
limitssection under theresourcessection. Theresourcessection must look like the following piece of code after the change:resources: requests: cpu: 100m memory: 256Mi -
Save the file and wait until all the
auth-idppods restart. The pods might take a few minutes to restart.