Troubleshooting AutoTrace webhook
If you notice that the Instana AutoTrace webhook is not working on your Kubernetes resources, troubleshoot the issues by using the following steps:
Verifying that the Instana AutoTrace webhook is receiving requests
To verify that the Instana AutoTrace webhook is receiving requests, check the logs of the instana-autotrace-webhook pod by running the following command:
kubectl logs -l app.kubernetes.io/name=instana-autotrace-webhook -n instana-autotrace-webhook
In a functioning installation, you can see the following logs:
14:41:37.590 INFO |- [AdmissionReview 48556a1a-7d55-497b-aa9c-23634b089cd1] Applied transformation DefaultDeploymentTransformation to the Deployment 'test-netcore-glibc/test-apps'
14:41:37.588 INFO |- [AdmissionReview 1d5877cf-7153-4a95-9bfb-de0af8351195] Applied transformation DefaultDeploymentTransformation to the Deployment 'test-nodejs-12/test-apps'
If you do not see such logs, your Kubernetes setup might have a problem. Continue troubleshooting by checking the following sections.
Checking the kube-apiserver logs
Check the logs of your kube-apiserver. These logs report on whether the Instana AutoTrace webhook is being started and provide information about the outcome of the execution.
If you identify that the AutoTrace webhook is not working as expected, use the following troubleshooting steps for the commonly observed issues.
Troubleshooting common issues
No network connectivity between kube-apiserver and the instana-autotrace-webhook pods
The most common issue is that the kube-apiserver cannot reach the worker nodes that are running the instana-autotrace-webhook pods. This issue might be caused by the security policies, which prevents the Instana AutoTrace
webhook from working.
To resolve this issue, modify your network settings so that kube-apiserver can access the instana-autotrace-webhook pods. Review your network security policies to make sure that kubeapi-server can initiate
connections and receive responses from instana-autotrace-webhook. Instana cannot provide direct guidance for resolving this issue because the solutions vary based on your policy and enforcement mechanisms.
kube-apiserver and the instana-autotrace-webhook pods cannot negotiate a TLS session
Sometimes, the cryptography restrictions (in terms of which algorithms can be used for TLS) prevent kube-apiserver from negotiating a TLS session with the instana-autotrace-webhook pod.
To resolve this issue, open a Support ticket, and inform IBM support about which cryptography algorithms your clusters support.
Insufficient memory
The AutoTrace webhook instruments applications by using LD_PRELOAD environment variable to load technology libraries before any other libraries. This approach uses additional memory, as the preloaded libraries are loaded into
the application's process. As a result, the container memory limit might need to be increased to accommodate for it.
If the pod does not have enough memory to load the instrumentation libraries, the pod can enter a restart loop with the following errors:
OOMKilledCrashLoopBackOff
To fix the restart loop, increase the memory limit in your deployment to account for the extra memory needed for the instrumentation of the application. You can increase the memory limit by updating the spec.template.spec.containers[x].resources.limits.memory field. Specifically, reference the following table for individual tracer memory overhead.
| Technology | Additional memory required |
|---|---|
| Ruby | 25 MB |
| Netcore | 100 MB |
| NGINX | 100 MB |
| Node.js | 70 MB |
| Python | 25 MB |