Troubleshooting AutoTrace webhook
If you notice that the Instana AutoTrace webhook is not working on your Kubernetes resources, troubleshoot the issues by using the following steps:
Verifying that the Instana AutoTrace webhook is receiving requests
To verify that the Instana AutoTrace webhook is receiving
requests, check the logs of the
instana-autotrace-webhook pod by running the following
command:
kubectl logs -l app.kubernetes.io/name=instana-autotrace-webhook -n instana-autotrace-webhook
In a functioning installation, you can see the following logs:
14:41:37.590 INFO |- [AdmissionReview 48556a1a-7d55-497b-aa9c-23634b089cd1] Applied transformation DefaultDeploymentTransformation to the Deployment 'test-netcore-glibc/test-apps'
14:41:37.588 INFO |- [AdmissionReview 1d5877cf-7153-4a95-9bfb-de0af8351195] Applied transformation DefaultDeploymentTransformation to the Deployment 'test-nodejs-12/test-apps'
If you do not see such logs, your Kubernetes setup might have a problem. Continue troubleshooting by checking the following sections.
Checking the kube-apiserver logs
Check the logs of your kube-apiserver. These logs
report on whether the Instana AutoTrace webhook is being started
and provide information about the outcome of the execution.
If you identify that the AutoTrace webhook is not working as expected, use the following troubleshooting steps for the commonly observed issues.
Troubleshooting common issues
No network connectivity between kube-apiserver and the instana-autotrace-webhook pods
The most common issue is that the kube-apiserver
cannot reach the worker nodes that are running the
instana-autotrace-webhook pods. This issue might be
caused by the security policies, which prevents the Instana
AutoTrace webhook from working.
To resolve this issue, modify your network settings so that
kube-apiserver can access the
instana-autotrace-webhook pods. Review your network
security policies to make sure that kubeapi-server can
initiate connections and receive responses from
instana-autotrace-webhook. Instana cannot provide
direct guidance for resolving this issue because the solutions vary
based on your policy and enforcement mechanisms.
kube-apiserver and the instana-autotrace-webhook pods cannot negotiate a TLS session
Sometimes, the cryptography restrictions (in terms of which
algorithms can be used for TLS) prevent kube-apiserver
from negotiating a TLS session with the
instana-autotrace-webhook pod.
To resolve this issue, open a Support ticket, and inform IBM support about which cryptography algorithms your clusters support.
Insufficient memory
The AutoTrace webhook instruments applications by using
LD_PRELOAD environment variable to load technology
libraries before any other libraries. This approach uses additional
memory, as the preloaded libraries are loaded into the
application's process. As a result, the container memory limit
might need to be increased to accommodate for it.
If the pod does not have enough memory to load the instrumentation libraries, the pod can enter a restart loop with the following errors:
-
OOMKilled -
CrashLoopBackOff
To fix the restart loop, increase the memory limit in your
deployment to account for the increased memory requirement for the
instrumentation of the application. You can increase the memory
limit by updating the
spec.template.spec.containers[x].resources.limits.memory
field. Specifically, reference the following table for individual
tracer memory overhead.
| Technology | Additional memory required |
|---|---|
| Ruby | 25 MB |
| Netcore | 100 MB |
| NGINX | 100 MB |
| Node.js | 70 MB |
| Python | 25 MB |