Slow interaction between kubelet and Docker cause PLEG issues
Docker delays its response to the kubelet when you create a pod.
Symptoms
Pods might fail to create because there is a timeout for kubelet to call the Docker engine.
Causes
There is a overload of pods on the host node. You might receive the following error message:
skipping pod synchronization - [PLEG is not healthy: pleg was last seen active 6m55.488150776s ago; threshold is 3m0s]
Reduce the possibility of occurrence
Update the kubelet service file.
-
Edit the kubelet service configuration file. Run the following command:
vi /etc/systemd/system/kubelet.service
-
Add the
--housekeeping-interval
option to increase the interval time. The default value is 10s. Add the following option:--housekeeping-interval=30s
Your kubelet service configuration file might resemble the following output:
[Unit] Description=Kubelet Service Documentation=https://github.com/kubernetes/kubernetes [Service] EnvironmentFile=-/etc/environment ExecStart=/opt/kubernetes/hyperkube kubelet \ --cadvisor-port=0 \ --docker-disable-shared-pid \ --bootstrap-kubeconfig=/etc/cfc/kubelet/kubelet-bootstrap-config \ --kubeconfig=/etc/cfc/kubelet/kubelet-config \ --cert-dir=/etc/cfc/kubelet \ --config=/etc/cfc/kubelet/kubelet-service-config \ --dynamic-config-dir=/etc/cfc/kubelet/kubelet-dynamic-config \ --network-plugin=cni \ --hostname-override=9.111.255.33 \ --node-ip=9.111.255.33 \ --pod-infra-container-image=hyc-cloud-private-edge-docker-local.artifactory.swg-devops.com/ibmcom-amd64/pause:3.1 \ --node-labels=node-role.kubernetes.io/worker=true, \ --register-with-taints= \ --keep-terminated-pod-volumes=false \ --housekeeping-interval=30s Restart=always RestartSec=10 [Install] WantedBy=multi-user.target
-
Restart the kubelet service on the host node with following commands:
systemctl daemon-reload systemctl restart kubelet
-
You must specify a resource request and limit for your application. For more details, see Setting resource quota.