Troubleshooting .NET tracing on Docker

Symptoms: Application shows metrics but no traces, some services report but others don't, or tracing stops after deployment or update.

Troubleshooting steps:

1. Verify that the .NET version of the application is supported:

   • .NET Core Runtime 5.0 or later
2. Verify that environment variables are present in the Dockerfile as per the configuration example in Setting up .NET tracing on Docker.
3. Verify that paths used for environment variables are valid, accessible by the process or application, and correct.
4. Verify that tracing is enabled in the agent configuration.yaml file.
5. Collect trace logs by using the methods described in Collecting logs.

6. Check the Instana tracing logs:

   • If the logs are empty or no traces are visible, the application is not producing spans.
   • If traces are present in the logs but are missing information (for example, host), the problem is with the .NET tracing.
   • If traces are present in the logs and contain information that is missing in the UI, the problem occurred either in the Instana backend or the agent.

Symptom: Application container crashes after adding the Instana agent but works when the agent is removed.

Troubleshooting steps:

1. Verify that the .NET version of the application is supported:

   • .NET Core Runtime 5.0 or later

2. Collect container logs:

   docker logs <container_name_or_id> > container.log

3. Collect dump files for further analysis:

   docker exec -it <container> \
   /usr/share/dotnet/shared/Microsoft.NETCore.App/*/createdump \
   -p 1 \
   -o /tmp/sampleapp.dmp

4. Verify whether any exceptions exist in the logs indicating that Instana could be causing an issue.

Complete the following steps:

1. Enable debug logs by adding the following environment variables to your Dockerfile:

   Note: Make sure that paths are valid, accessible by the process or application, and correct.

   ENV INSTANA_LOG_SPANS=1
   ENV INSTANA_DEBUG_TRACER=1
   ENV INSTANA_NET_LOG_PATH="<Log_path>"
   ENV INSTANA_NET_LOG_LEVEL=DEBUG
   ENV INSTANA_TRACER_ENTEREXIT_LOGGING=1
   ENV INSTANA_EXTENDED_DEBUG=1
   ENV INSTANA_CLRLOG_PATH=/var/tmp/clr_

2. Restart the application and produce calls.

3. Collect and investigate the following logs:

   • CLR debug logs
   • Instana agent debug logs
   • Instana agent traces logs
   • Docker container logs

4. To collect Instana agent logs, complete the following steps:

   1. Stop the agent:

      systemctl stop instana-agent.service

   2. Remove the logs: Delete all files in the log directory to start fresh by running the following command:

      sudo rm -rf /data/logs/*

   3. Set log level to Debug: Change the log level severity in the agent configuration.yaml file. For more information, see Setting up the agent configuration file.
   4. Start the agent.
   5. Reproduce the issue: Allow the issue to occur and let the agent run for approximately 15 minutes.

   The Instana agent logs to the console, which is managed by Docker and is accessible by using the Docker logs.

5. To collect Instana trace logs, complete the following steps:

   1. Go to [agent-dir]\etc\instana and open the file com.instana.agent.main.sender.File.cfg.

   2. Add the following lines in the file:

      prefix=instanaTraces
      type=traces

      This configuration provides you with a log file containing all the traces in [agent-dir]/data/log with the instanaTraces prefix.

   3. Restart the Instana agent.
   4. Restart the application.
   5. Allow the application to run for approximately 15 minutes.

   6. Collect logs from the following paths:

      • Path specified by the variable INSTANA_NET_LOG_PATH
      • instana-agent-installation-folder/data/log

6. To collect Docker container logs, run the following command:

   docker logs <container_name_or_id> > container.log

To collect the .NET Tracer logs and CLR debug logs, use a Docker bind mount with an existing host path.

Complete the following steps:

1. Ensure that you have a valid host path with write permissions for the container.

2. Run your Docker container with the Bind Mount and the .NET Tracer and CLR debug log path environment variables pointed to the mapped path as shown in the following example:

   Note: Make sure that paths are valid, accessible by the process or application, and correct.

   docker run -d \
       -p 8080:8080 \
       -v /var/log/instana:/var/tmp \
       --env=INSTANA_CLRLOG_PATH="/var/tmp/clr_" \
       --env=INSTANA_NET_LOG_PATH="/var/tmp" \
       --name sampleapp \
       myuser/sampleapp:v2.0

3. Restart the application and produce calls.
4. Collect logs from the host path /var/log/instana.

• Setting up .NET tracing on Docker
• Review known issues and limitations