Istio-pilot pods have high CPU usage
Symptoms
istio-pilot
pods have high CPU usage.- Frequent 503 Service Unavailable error from
istio-ingressgateway
in the Ingress pod logs - Almost all microservices are down.
Resolving the problem
Set the PILOT_PUSH_THROTTLE
and PILOT_DEBOUNCE_AFTER
environment variables to reduce the CPU usage for istio-pilot
. The variables need to be set in the env:
section of container discovery
in deployment istio-pilot
.
Complete the following steps:
-
Create
istio-patch.yaml
with the following content:spec: template: spec: containers: - name: discovery env: - name: PILOT_PUSH_THROTTLE value: "1" - name: PILOT_DEBOUNCE_AFTER value: "10s"
-
Run the
patch
command to update the deployment:kubectl --namespace=istio-system patch deployment istio-pilot --patch "$(cat istio-patch.yaml)"
After running the
patch
command, theenv:
section of containerdiscovery
in deploymentistio-pilot
should include these values:- name: PILOT_PUSH_THROTTLE value: "1" - name: PILOT_DEBOUNCE_AFTER value: 10s