- Configuring OpenTelemetry data ingestion
- Sending OTLP data to Instana agent
- Sending OTLP data to Instana backend
- Sending native data to Instana backend by using Instana Exporter
- Using OpenTelemetry Contrib Collector
- Analyzing OpenTelemetry data
- Development considerations
- OpenTelemetry clients
Instana provides support for some components of OpenTelemetry such as tracing. Some components are in public beta or will be released soon. Work is in progress to make OpenTelemetry a first-class citizen in Instana.
Instana can seamlessly integrate with OpenTelemetry and can also enhance the data from OpenTelemetry. This integration further enhances data acquisition capabilities.
Instana supports most OpenTelemetry Resource Semantic Conventions.
OpenTelemetry resources and Infrastructure correlations
Instana supports the Resource attributes of OpenTelemetry. Instana can enhance OpenTelemetry resources when an Instana agent is running on the same host. To make the Instana OpenTelemetry resource detector work, you need to deploy an Instana agent, and make the agent running on the same host.
Instana OpenTelemetry resource detectors enrich OpenTelemetry tracing information with additional resource attributes. The additional attributes help to establish a link between the tracing data and the underlying infrastructure entities.
The following fields are attached to the resource:
process.pid: The PID of the process in the root PID namespace.
instana.host.id: The UUID of the Instana agent on this host.
service.instance.id: The ID of the OpenTelemetry service. This ID is used to correlate tracing data with the process entity.
For detailed information on infrastructure correlation on Linux hosts, see agent's HTTP API endpoint.
See the following available Instana OpenTelemetry Resource detectors:
Work is in progress to make OpenTelemetry fully interoperable with Instana's own AutoTrace technology
Instana can seamlessly integrates with OpenTelemetry traces. Instana correlates OpenTelemetry traces by using Instana-generated traces that use the default W3C Trace Context.
Instana supports most OpenTelemetry Trace Semantic Conventions.
linksare not supported.
- Mixing Instana default context propagator with context propagators other than
W3C Trace Contextis not supported.
W3C Trace Contextis the default context propagator of OpenTelemetry.
OpenTelemetry metrics appear as "OpenTelemetry Custom Metrics" associated with the agent, host or process from where they are collected. OpenTelemetry entities can be found under Infrastructure Analytics under the type OpenTelemetry.
For "Event & Alerts" or the Infrastructure Map, you can query OpenTelemetry custom metrics using the "Dynamic Focus" with
The sensor collects the following metric types:
Collection of OpenTelemetry logs is still under development and test. The host agent will show some benign exceptions if it receives OpenTelemetry logs data.
Configuring OpenTelemetry data ingestion
To ingest data to Instana from applications or systems where OpenTelemetry is enabled, use the following 4 ways:
- Applications or systems can use OTLP protocol to send data to Instana agent. Then, Instana agent forwards the data to Instana backend.
- Applications or systems can use the OTLP protocol to send data directly to Instana backend.
- Applications or systems can use the OTLP protocol to send data to the OpenTelemetry Contrib Collector, which then forwards the data to Instana backend with its Instana Exporter.
- Applications or systems can load related language-based Instana Exporter into their code to work for the corresponding language-based OpenTelemetry SDK. The Instana Exporter can use the native Instana protocol to send data directly to the serverless acceptor of the Instana backend.
Sending OTLP data to Instana agent
By using the host agent as a proxy, OpenTelemetry can send tracing and metric data to the Instana agent. Then, the data is sent to the Instana backend for processing.
The OpenTelemetry ingestion in the host agent is deactivated by default to not interfere with other collectors on the host. To enable the reception of OpenTelemetry data in the host agent, add the following snippet to your host agent's
com.instana.plugin.opentelemetry: enabled: <boolean> # legacy setting, will only enable grpc, defaults to false grpc: enabled: <boolean> # takes precedence over legacy settings above, defaults to true if "grpc:" is present http: enabled: <boolean> # allows to enable http endpoints, defaults to true if "http:" is present
After the configuration is activated, the host agent will automatically activate a collector for OpenTelemetry's default OTLP (OpenTelemetry protocol) exporter on port
4317 for gRPC and on port
4318 for HTTP or HTTPS
Sending OTLP data to Instana agent in Kubernetes
The Instana Agent Service provided on Kubernetes via the Instana Agent Helm Chart is very useful in combination with the OpenTelemetry support. It ensures that the data is forwarded to the Instana Agent which runs on the same Kubernetes node. The Instana Agent Service is a Kubernetes service that can be used as the domain name of the pod where Instana agent is running. This allows the Instana Agent to populate the infrastructure correlation data correctly.
To enable the default OTLP ports 4317 for gRPC and 4318 for HTTP or HTTPS connections, set the following flags to
- If you use Helm Chart deployment, set the following Helm flags to
- If you installed the Instana host agent by using the Operator, then you can enable the OTLP through the Instana agent custom resource. For more information, see Installing the Host Agent on Kubernetes.
Set the OTLP exporter endpoint by using the following format:
To get the host IP, use the Downward API snippet to set an environment variable to the application pod. For more information about setting an environment variable to the application pod, see Network Access configuration.
Enabling TLS encryption for OpenTelemetry ingestion
You can enable TLS encryption on the host agent, so that all data that is sent to the OpenTelemetry ingestion endpoint is TLS-encrypted. For more information, see Enabling TLS Encryption.
Avoiding compatibility issues caused by multiple tracers
Some tracers might have conflict to work together. For example, Instana agent does not automatically monitor JVMs that also have a number of Java agents running including OpenTelemetry Instrumentation for Java.
Another example is IBM ACE. You can use different ways for tracing. The new native solution is using OpenTelemetry, but you need to avoid using multiple tracing solutions together. For more information, see IBM ACE documents.
Sending OTLP data to Instana backend
Applications or systems can use the OTLP protocol to send data directly to Instana backend. The special endpoint of
otlp-acceptor component is used. Make sure that the
otlp-acceptor component of your Instana backend
is enabled. You also need to set required Instana key for validation. See the following example configuration of using the OpenTelemetry Collector, which is used to connect to Instana
exporters: otlp: endpoint: otlp-xxxx-saas.instana.io:4317 headers: x-instana-key: xxxxxxx
Note: Set the x-instana-key field with the key of Instana agent for the targeting Instana backend. Use the correct hostname of your Instana backend for the endpoint field. The self-hosted Instana backend can have any host name, and it doesn't have to be "otlp-xxxx-saas.instana.io".
Sending native data to Instana backend by using Instana Exporter
You need to load the corresponding language-based Instana Exporter into your applications to work for the corresponding language-based OpenTelemetry SDK. Though more and more people in the OpenTelemetry community choose the native OTLP protocol, this simple solution is still workable. Here are available Instana OpenTelemetry Exporters:
Using OpenTelemetry Contrib Collector
Applications or systems can use OTLP protocol to send data to the OpenTelemetry Contrib Collector. Then, the OpenTelemetry Contrib Collector forward the data to Instana backend severless acceptor by using its Instana Exporter. To send data to Instana backend serverless acceptor, configure your host agent's
configuration.yaml file as follows:
exporters: instana: endpoint: "https://xxxxxx/serverless" agent_key: "xxxxxxxxxx"
Set the agent_key field with the key of Instana agent. Note: The "Content-Length" limit of the serverless-acceptor is 50000000. If the request breaks the limit, you can see a "413 Request Entity Too Large" error. So, you also need to change the collector configuration file of the otel contrib collector: 1. Create a special instance of batch processor to be used along with the instana exporter. 2. Set the send_batch_max_size parameter of the batch processor to a proper value (by default no limit). For more information, see the readme file.
Analyzing OpenTelemetry data
Note: The following description is based on the assumption that the user has installed Instana backend version 239 or later.
Analyzing OpenTelemetry calls
We are hard at work to make OpenTelemetry a first-class citizen with full interoperability with Instana's own AutoTrace technology. So, there is almost nothing different if all spans or partial spans of a trace are generated by applications or systems enabled with OpenTelemetry, comparing with all traces coming from native Instana tracers. The same way of Call analytics can be used.
A typical method to filter calls from OpenTelemetry is to set the filer query to be the OpenTelemetry Resource attribute "
service.name" and "
is present". This is because the OpenTelemetry Resource
service.name" is required by OpenTelemetry specification for resource semantic conventions. OpenTelemetry
has various ways to specify this service name as described in the OpenTelemetry official documents.
You can use multiple resource and trace attributes to map Instana Service by using the default Service rules. As mentioned previously, the OpenTelemetry Resource attribute
service.name is very frequently used to be mapped as local Instana Service. You can also use other attributes as follows:
- OpenTelemetry Trace span attribute
peer.serviceis used to be mapped as the remote Instana Service.
- OpenTelemetry Trace span attribute
messaging.destinationis used to be mapped as the remote Instana Service based on the exiting
call.messaging.addressrule by using an internal conversion.
If you are an OpenTelemetry developer, and you want to create OpenTelemetry-enabled applications or systems with Instana, you need to consider following topics:
Note: Instana fully supports OpenTelemetry standard. All of the following components are optional and just for considerations of advanced integrations.
Propagators translate between vendor-specific trace correlation headers and the OpenTelemetry trace context. If an incoming HTTP request contains the proprietary Instana trace correlation headers (
the Instana OpenTelemetry propagator creates an OpenTelemetry trace context with the trace ID and span ID from those headers. The OpenTelemetry tracer can continue the trace from that trace context. Also, if an OpenTelemetry trace context
exists, the Instana OpenTelemetry propagator will use it to inject the proprietary Instana trace correlation headers into outgoing requests.
Note: Instana tracers support the W3C trace context specification, which is the default correlation header format used by OpenTelemetry. Therefore you usually don't need to add an Instana OpenTelemetry propagator to your application. For more information about the exact level of support, see W3C Trace Context Support.
Available Instana OpenTelemetry propagators
- Node.js: @opentelemetry/propagator-instana
An OpenTelemetry sampler makes a sampling decision, that is, it decides whether tracing data is captured for a particular request or not. Instana OpenTelemetry samplers make this decision based on the
sampled flag of
the trace context. If the value of the
sampled flag is not
false, the sampler will decide to capture the request.
Available Instana OpenTelemetry samplers
- Node.js: @instana/opentelemetry-sampler
You can include OpenTelemetry Instrumentation in your applications by using language-based OpenTelemetry SDK. Then, the data can be sent to Instana over the OTLP protocol.
Currently, only OTel C++ client SDK is supported.
OTel C++ client SDK package
The OTel C++ client SDK package is an OpenTelemetry OTLP exporter for C++. The package is built on Opentelemetry-cpp. For more information, see Installing the OTel C++ client SDK package.