Monitoring .NET Framework

The .NET sensor is automatically deployed and installed after you install the Instana agent.

Note: This page is about our support for .NET Full Framework based applications. If you are looking for support on .Net or .NET Core, see Monitoring .NET and .NET Core based applications.

Supported information

Supported operating systems

Only Windows® is supported for monitoring .NET Framework. The supported Windows operating systems are the same as the Windows agent requirements.

Supported versions

Instana supports monitoring applications that run on .NET Framework 4.5.1 and later versions.

Runtime Support status
.NET Framework 4.5.1 and later GA

Supported libraries and frameworks for tracing

HTTP Version
ASP.NET MVC >= 5.0, >= 5.1, >=5.2.2
ASP.NET WebAPI >= 5.0, >= 5.1, >=5.2.2
ASP.NET WebForms >= 4.5.2
ASP.NET Core 1.0.2, 1.0.3, 1.1.2, 1.1.3, 1.1.7, 2.0.0, 2.0.1, 2.0.2, 2.2.0
ServiceStack 4.4, 4.5
System.Net.Http.HttpClient >= 4.5.2
System.Net.HttpWebRequest >= 4.5.2
Databases Version
Microsoft SQL-Server (System.Data.SqlClient) >= 1
Oracle ODP (Oracle.DataAccess.Client) 2.102.5, 2.112.1, 4.112.2, 4.112.3, 4.121.2
Oracle Devart (Oracle.ManagedDataAccess.Client) 4.121.2.0 (v12.1.x), 4.122.1.0 (v12.2.x), 4.122.18.3 (v18.x), 4.122.19.1 (v19.x), 4.122.21.1 (v21.x)
PostgreSQL (NpgSql) >= 2
Cassandra 3.4.0, 3.7.0
Couchbase 3.1.7
Elastic Search 6.0.0
MongoDB >= 1.8.0, 2.4.4, >= 2.10.0
Redis Service Stack >= 5.0
Redis Stack Exchange >= 2.0
Memcached 2.11.0, 2.16.0
IBM DB2 >= 11.1.0.4
Sql (System.Data) >= 4.5.2
ORM Version
Entity Framework >= 4.5.2
Remoting Version
Windows Communication Foundation Services >= 4.5.2
.NET Remoting >= 4.5.2
Messaging Version
Azure Queue Storage 12.8.0
Microsoft Message Queue >= 4.5.2
EasyNetQ 3.3.9
RabbitMQ 3.6.9, 4.1.0, 5.0.1
Logging Version
NLog >= 4
Log4Net 1.2.11, 1.2.14, 1.2.15, 2.0.7, >= 2.0.8
Serilog >= 2.8.0

For logging, you can set the environment variable INSTANA_LOG_LEVEL_CAPTURE to prevent creating spans for logs with lower level than the one set. The value can be WARN, ERROR or OFF for disabling this option. The default value is WARN.

Schedulers / Task-Management Version
Hangfire 1.6.20
Quartz 3.0.7

Sensor Data Collection

Tracked Configuration Metrics
Name GC Activity
Version Memory Usage
Arguments Thread-locks
Contention

Health Signatures

For every sensor, a curated knowledgebase of health signatures is evaluated continuously against the incoming metrics. Based on the evaluation, issues or incidents are raised depending on the user impact.

Built-in events trigger issues or incidents based on failing health signatures on entities, and custom events trigger issues or incidents based on the thresholds of an individual metric of any given entity.

For information about built-in events for the .NET sensor, see the Built-in events reference.

InstanaPCP and performance counters

For more information on InstanaPCP and performance counters, see Monitoring Windows processes page.

Tracing

This page describes how to setup .NET Framework Tracing for your environment and how it works.

How Tracing Works

When you enable tracing by changing the configuration file, tracing data is collected for all applications that use the supported technologies. .NET Framework-Tracing follows the same scheme as all other tracing packages. Distributed traces in polyglot applications are automatically collected, and they are displayed in Instana UI.

When an application with tracing enabled starts up, you might find the startup-time to be increased in comparison to startup without tracing. This is due to the profiler attaching, inspecting the classes contained in the referenced assemblies, and then just-in-time compiling some methods' code for instrumentation. After the instrumentation has been applied, the impact on the runtime performance of your applications is extremely low, as with any other tracing-package that Instana provides.

The tracing component setup is a mostly automatic function of the agent. However, there is a manual step involved, which we describe in the following sections.

Setting up Tracing for .NET Framework

When the Instana agent starts on your machine that runs on Windows, the Instana agent install the tracing extension for .NET Framework. The extension is composed of several files which contain the actual tracing components. Because Instana uses the Profiling API defined in the .NET framework, it involves COM-Libraries and managed assemblies. When tracing is enabled, the files are registered appropriately so that all applications can use them for tracing. The sensor and tracing-extensions are enabled by default.

Upon startup of the agent, InstanaPCP will log information about the process of trace-enablement. This will look like this (log lines shortened):

The following example shows the configuration.yaml with .NET Framework tracing enabled

INFO  | com.instana.agent - 1.1.326 | Writing location for instrumentation-advices to registry
INFO  | com.instana.agent - 1.1.326 | Setting up tracing-environment
INFO  | com.instana.agent - 1.1.326 | Registering native profiler for .NET
INFO  | com.instana.agent - 1.1.326 | Agent UUID received:ac:bc:32:ff:fe:90:7d:bb
INFO  | com.instana.agent - 1.1.326 | Registering System.Collections.Immutable.dll in GAC
INFO  | com.instana.agent - 1.1.326 | .NET Tracing is enabled
INFO  | com.instana.agent - 1.1.326 | Registering instrumentation-DLLs in GAC
INFO  | com.instana.agent - 1.1.326 | Registering Instana.ManagedTracing.dll in GAC
INFO  | com.instana.agent - 1.1.326 | Setting environment-variables for .NET tracing
INFO  | com.instana.agent - 1.1.326 | Registry-Key for WAS-Environment does not exist, will create now
INFO  | com.instana.agent - 1.1.326 | Setting environment for WAS
WARN  | com.instana.agent - 1.1.326 | You should consider restarting the service WAS in order to make it use the new profiler. Already running application will not use it until restarted.
INFO  | com.instana.agent - 1.1.326 | Registry-Key for W3SVC-Environment does not exist, will create now
INFO  | com.instana.agent - 1.1.326 | Setting environment for W3SVC
WARN  | com.instana.agent - 1.1.326 | You should consider restarting the service W3SVC in order to make it use the new profiler. Already running application will not use it until restarted.
INFO  | com.instana.agent - 1.1.326 | Setting memoryUsageExcessThreshold set to configured value of 100000000
INFO  | com.instana.agent - 1.1.326 | Self-Monitoring for Windows-Extensions has been started successfully
INFO  | com.instana.agent - 1.1.326 | Setting memoryUsageWarningThreshold set to configured value of 80000000
INFO  | com.instana.agent - 1.1.326 | Starting tracing-session
INFO  | com.instana.agent - 1.1.326 | Windows-Extension REST-Endpoint has been setup successfully at http://localhost:42657
INFO  | com.instana.agent - 1.1.326 | Registering Instana.ManagedTracing.Modules.dll in GAC
INFO  | com.instana.agent - 1.1.326 | Instana ETW-Reader setup and listening

The extension registers a native COM-DLL (profiler), along with 3 managed assemblies, which contain the actual instrumentation code or are required for the instrumentation to work (Instana.ManagedTracing.dll, Instana.ManagedTracing.Base.dll, System.Collections.Immutable.dll). After registering the components, the extension applies changes to the registry, so that any instances of W3SVC or WAS will attach the profiler the next time they are being started. This is why you will find a warning in the logs regarding restarting already running applications. Instana does not restart these services automatically, as we do not want to interrupt your services.

Enabling tracing for IIS-hosted applications

In case you are running IIS hosted applications, there is no need for any additional setup, InstanaPCP will take care of everything, yours is just to run the IIS reset after the Instana agent is running and InstanaPCP application is running. You can check if InstanaPCP is running by using the Task Manager for Windows.

Note: If you don't run the IIS reset, IIS workers don't know that they need to be instrumented, and you are not able to trace IIS-hosted applications.

Enabling Tracing for Non-IIS Hosted Applications

The Instana Agent does not enable tracing for every .NET based application. This is on purpose, since it might produce undesired component behavior like tracing Windows system applications that are not in scope of your monitoring needs.

Setting up env variables for Processes

While IIS and WAS get configured to attach the profiler automatically by setting the appropriate environment-variables, standalone-processes are not. You can enable tracing for these processes by providing the necessary environment-variables by either setting them on the user level (when you use specific technical users which run your applications), or by setting the environment-variables before starting the actual process.

To enable tracing for a specific process you have to provide the following environment-variables with their respective values:

Environment variables needed for tracing

SET COR_ENABLE_PROFILING=1
SET COR_PROFILER={FA8F1DFF-0B62-4F84-887F-ECAC69A65DD3}

You can put these settings into a batch or powershell-script which starts your processes after setting the env-vars.

Setting up env variables Windows-Services

If you are running Windows-Services you can also put these settings into the registry, on a per-service basis, which gives you better control over which services to trace.

Go to the Registry Editor on the path HKLM\System\CurrentControlSet\Services\<ServiceName>, add a MULTI_SZ value by the name of Environment (if it doesn't already exist), and then copy the variables COR_ENABLE_PROFILING=1 and COR_PROFILER={FA8F1DFF-0B62-4F84-887F-ECAC69A65DD3} into that value.

To correctly set the contents of a MULTI_SZ value, put one key-value-pair per line like displayed as follows:

COR_ENABLE_PROFILING=1
COR_PROFILER={FA8F1DFF-0B62-4F84-887F-ECAC69A65DD3}

Hit enter after last line.

Custom Tracing

Instana's .NET Tracing SDK enables to your own custom tracing in situation where our automatic instrumentation does not apply (i.e., you are using components we are not automatically supporting).

Disabling Tracing for .NET Framework

When you want to stop tracing of .NET Framework applications for whatever reason, you can simply do so by changing the configuration file back to its initial state, or by just setting the corresponding flag to "false."

configuration.yaml with .NET Framework-Tracing disabled

# CLR Tracing
    com.instana.plugin.clr:
      tracing:
        enabled: false

After changing the config back, you will have to restart the agent for the changes to take effect.

Due to the nature of how the Profiling API works, disabling tracing in the agent will not remove / detach the profiler from processes that have been started with tracing enabled (i.e. your worker-processes). To make sure the profiler is not loaded into the application, you have to restart the host-process.

Open Telemetry Integration

Instana provides additional .NET components for integrating with OpenTelemetry, which are described on the page Open Telemetry Integration.

Troubleshooting

There might be cases where your setup does not work at first. If this troubleshooting section does not answer the questions you have, contact our support team and let us know, so that we can help you and update our documentation accordingly.

It's always the best to first check the steps we needed to perform in order to enable tracing one more time. So - Focus on one application first. Try to get that one monitored, and then instrumented.

  • Depending on the setup always look again for the details provided above on how to enable monitoring.
  • After the agent starts, and it is running for a few minutes, you will see in Task Manager InstanaPCP.exe, if you see it and your application is IIS hosted, then do the iisreset so you can be sure that the instrumentation is attached to the applications hosted by IIS. If you application is not IIS hosted, you then don't need to perform iisreset command.
  • If the app is IIS hosted and iisreset is done look for the application in the UI - it should be there by the mark of one minute.
  • Try making some requests to that application, and then try to search for that service in UI (easiest is to find that process on that host, and from there to go to the service, and then click on Analyze traces).
  • If this doesn't work try turning on agent tracing logs - first set them to capture only metrics and you will see if the sensor has even found that application in the list of metrics that it sends to the backend. You can do the same by comparing the agent logs with the process id of that application, you will see the log saying that a CLR sensor has been created for a given process id.
  • If you find those metrics or logs, then enable tracing logs for the agent - it will capture traces that it sends to the backend - those will contain the process id and name, so try to match those with the application you are trying to instrument.

Environment variable not defined

Monitoring issue type: clr_env_var_not_defined

One or more environment variables, which are necessary to activate the tracing of .NET Framework application, are not set correctly.

If this happens for applications hosted on IIS, it might be that the iisreset has not been issued at all, or before the agent could register the libraries needed for tracing. As a solution, issue an iisreset from an elevated prompt.

If this does not help, consider restarting the machine, which might be needed depending on the version of Windows you are running and whether there were (un-)installations happening before running the instana agent.

For applications not hosted on IIS, refer to the Enabling Tracing for Non-IIS Hosted Applications section for the list of environment variables and the respective values you need to set for tracing your .NET Framework applications.

Environment variable has an invalid value

Monitoring issue type: clr_env_var_invalid_value

At least one environment variable that is necessary to activate the Instana .NET Framework tracing, does not have the right value. (Depending on which environment variables are missing, you may still see metrics in the Instana dashboards for your .NET Framework application, but definitely not traces.) Please refer to the Enabling Tracing for Non-IIS Hosted Applications section for the list of environment variables and the respective values you need to set for tracing your .NET Framework applications with Instana.

Also, it might be the case that the reason for the environment variable to have the wrong value is the presence of another APM tool. (In terms of hooking into the .NET runtime, the mechanisms the various APM use are pretty similar.) If we recognize this situation, you will see it in the message in the Instana user interface. Before Instana can trace your .NET Framework application, you will have turn off the other APM agent, at least insofar .NET Framework monitoring is concerned; for how to do that, refer to the other APM tool's documentation.

InstanaPCP not running

Monitoring issue type: clr_instana_pcp_not_running

You might find warning messages in your agent's log, stating that InstanaPCP does not seem to be running. This message is triggered when the agent tries to interact with InstanaPCP but fails to do so. Usually, the agent starts InstanaPCP early in the bootstrapping-process and can then communicate with it. Under the following circumstances, InstanaPCP might not be able to start:

  • The agent is not running with sufficient privileges (not running as Local System or Administrator).
  • .NET framework version on the host is not supported. Only .NET framework 4.5.1 and later versions are supported.
  • InstanaPCP restarted itself due to memory overconsumption.

If you see that this message appears repeatedly, complete the following checks:

  1. Make sure that your agent runs with a user that has local administrative privileges. If not, use a different user account to run the agent, or grant the user with local administrative privileges. When you install the Instana agent from the installation package, the service is normally run as Local System, which gives the service sufficient privileges.

  2. Check which version of the .NET Framework is installed on your machine. The Windows installer package automatically installs the minimum version of the .NET Framework that InstanaPCP needs. However, if you install the host agent from a ZIP-archive, you might need to install the framework yourself. If you are unsure if you have the correct version, try to run InstanaPCP manually from a command line by locating it in the <agent-dir>\data\repo\com\instana\agent-windows-extensions\<latest>\agent-windows-extensions-<latest> directory. If you do not have the correct version of the .NET Framework installed, an associated prompt is displayed.

  3. Check whether you see messages that point to over consumption in the log as follows:

    InstanaPCP.exe Private-Bytes is above warning-treshhold for XXX times now
    
    Killing InstanaPCP.exe for excessive memory-usage. Possible memory-leak in native Windows-Libraries.
    

    These messages are caused by either of the following two issues:

    1. The Windows machine where InstanaPCP is running has an old version of IIS7 and has a specific memory leak. This leak can be patched by installing a Microsoft update. This problem might happen on Windows 2008 machines. Before you install this Microsoft update, contact Instana Support to make sure that the memory-leak issue is happening on your machine. The support team can then provide you with the link to update your Microsoft package.

    2. Many .NET applications are running on the server, and those applications are generating many traces. In this case, increase the memory limits for InstanaPCP by applying changes to the following settings in your agent configuration.yaml file:

      com.instana.plugin.clr:
        pcp_warn_limit: <MB-treshold before warning>
        pcp_stop_limit: <MB-treshold  to stop / restart the process>
      

InstanaPCP not connected

Monitoring issue type: clr_instana_pcp_not_connected

The host agent cannot connect to the InstanaPCP process, which prevents the retrieval and processing of tracing data and runtime metrics for .NET processes.

This is usually a transient failure, which resolves itself quickly after the InstanaPCP finishes initializing. If not, contact Instana support.