OpenTelemetry

OpenTelemetry resources and Infrastructure correlations

Instana supports the Resource attributes of OpenTelemetry. Instana can enhance OpenTelemetry resources when an Instana agent is running on the same host. To make the Instana OpenTelemetry resource detector work, you need to deploy an Instana agent, and make the agent running on the same host.

Instana OpenTelemetry resource detectors enrich OpenTelemetry tracing information with additional resource attributes. The additional attributes help to establish a link between the tracing data and the underlying infrastructure entities.

The following fields are attached to the resource:

• process.pid: The PID of the process in the root PID namespace.
• instana.host.id: The UUID of the Instana agent on this host.
• service.instance.id: The ID of the OpenTelemetry service. This ID is used to correlate tracing data with the process entity.

For detailed information on infrastructure correlation on Linux hosts, see agent's HTTP API endpoint.

See the following available Instana OpenTelemetry Resource detectors:

• Node.js: @opentelemetry/resource-detector-instana

OpenTelemetry Tracing

Work is in progress to make OpenTelemetry fully interoperable with Instana's own AutoTrace technology

Instana can seamlessly integrates with OpenTelemetry traces. Instana correlates OpenTelemetry traces by using Instana-generated traces that use the default W3C Trace Context.

Instana supports most OpenTelemetry Trace Semantic Conventions.

OpenTelemetry metrics

OpenTelemetry metrics appear as "OpenTelemetry Custom Metrics" associated with the agent, host or process from where they are collected. OpenTelemetry entities can be found under Infrastructure Analytics under the type OpenTelemetry. For "Event & Alerts" or the Infrastructure Map, you can query OpenTelemetry custom metrics using the "Dynamic Focus" with entity.type:opentelemetry or entity.type:otel.

The sensor collects the following metric types:

• Gauges
• Histograms
• Summaries

OpenTelemetry logs

Collection of OpenTelemetry logs is still under development and test. The host agent will show some benign exceptions if it receives OpenTelemetry logs data.

Sending OTLP data to Instana agent

By using the host agent as a proxy, OpenTelemetry can send tracing and metric data to the Instana agent. Then, the data is sent to the Instana backend for processing.

The OpenTelemetry ingestion in the host agent is deactivated by default to not interfere with other collectors on the host. To enable the reception of OpenTelemetry data in the host agent, add the following snippet to your host agent's configuration.yaml:

com.instana.plugin.opentelemetry:
  enabled: <boolean>   # legacy setting, will only enable grpc, defaults to false
  grpc:
    enabled: <boolean> # takes precedence over legacy settings above, defaults to true if "grpc:" is present
  http:
    enabled: <boolean> # allows to enable http endpoints, defaults to true if "http:" is present

After the configuration is activated, the host agent will automatically activate a collector for OpenTelemetry's default OTLP (OpenTelemetry protocol) exporter on port 4317 for gRPC and on port 4318 for HTTP or HTTPS connections.

Sending OTLP data to Instana backend

Applications or systems can use OTLP protocol to send data directly to Instana backend. The special endpoint of the OTLP serverless accepter is used. Make sure that you have enabled the OTLP serverless accepter of your Instana backend by configuring your host agent's configuration.yaml as follows:

exporters:
  otlp:
    endpoint: otlp-xxxx-saas.instana.io:4317
    headers:
      x-instana-key: xxxxxxx

Note: Set the x-instana-key field with the key of Instana agent for the targeting Instana backend. Use the correct hostname of your Instana backend for the endpoint field. The self-hosted Instana backend can have any host name, and it doesn't have to be "otlp-xxxx-saas.instana.io".

Sending native data to Instana backend by using Instana Exporter

You need to load the corresponding language-based Instana Exporter into your applications to work for the corresponding language-based OpenTelemetry SDK. Though more and more people in the OpenTelemetry community choose the native OTLP protocol, this simple solution is still workable. Here are available Instana OpenTelemetry Exporters:

• Serverless Node.js Instana OpenTelemetry exporter: @instana/opentelemetry-exporter
• Serverless .NET Instana OpenTelemetry exporter: OpenTelemetry.Exporter.Instana
• Serverless Go Instana OpenTelemetry exporter: github.com/instana/go-otel-exporter

Using OpenTelemetry Contrib Collector

Applications or systems can use OTLP protocol to send data to the OpenTelemetry Contrib Collector. Then, the OpenTelemetry Contrib Collector forward the data to Instana backend severless acceptor by using its Instana Exporter. To send data to Instana backend serverless acceptor, configure your host agent's configuration.yaml file as follows:

exporters:
  instana:
    endpoint: "https://xxxxxx/serverless"
    agent_key: "xxxxxxxxxx"

Set the agent_key field with the key of Instana agent. Note: The "Content-Length" limit of the serverless-acceptor is 50000000. If the request breaks the limit, you can see a "413 Request Entity Too Large" error. So, you also need to change the collector configuration file of the otel contrib collector: 1. Create a special instance of batch processor to be used along with the instana exporter. 2. Set the send_batch_max_size parameter of the batch processor to a proper value (by default no limit). For more information, see the readme file.

Analyzing OpenTelemetry calls

We are hard at work to make OpenTelemetry a first-class citizen with full interoperability with Instana's own AutoTrace technology. So, there is almost nothing different if all spans or partial spans of a trace are generated by applications or systems enabled with OpenTelemetry, comparing with all traces coming from native Instana tracers. The same way of Call analytics can be used.

A typical method to filter calls from OpenTelemetry is to set the filer query to be the OpenTelemetry Resource attribute "service.name" and "is present". This is because the OpenTelemetry Resource attribute "service.name" is required by OpenTelemetry specification for resource semantic conventions. OpenTelemetry has various ways to specify this service name as described in the OpenTelemetry official documents.

Service mapping

You can use multiple resource and trace attributes to map Instana Service by using the default Service rules. As mentioned previously, the OpenTelemetry Resource attribute service.name is very frequently used to be mapped as local Instana Service. You can also use other attributes as follows:

• OpenTelemetry Trace span attribute peer.service is used to be mapped as the remote Instana Service.
• OpenTelemetry Trace span attribute messaging.destination is used to be mapped as the remote Instana Service based on the exiting call.messaging.address rule by using an internal conversion.

OpenTelemetry Propagators

Propagators translate between vendor-specific trace correlation headers and the OpenTelemetry trace context. If an incoming HTTP request contains the proprietary Instana trace correlation headers (X-INSTANA-T and X-INSTANA-S), the Instana OpenTelemetry propagator creates an OpenTelemetry trace context with the trace ID and span ID from those headers. The OpenTelemetry tracer can continue the trace from that trace context. Also, if an OpenTelemetry trace context exists, the Instana OpenTelemetry propagator will use it to inject the proprietary Instana trace correlation headers into outgoing requests.

Note: Instana tracers support the W3C trace context specification, which is the default correlation header format used by OpenTelemetry. Therefore you usually don't need to add an Instana OpenTelemetry propagator to your application. For more information about the exact level of support, see W3C Trace Context Support.

OpenTelemetry Samplers

An OpenTelemetry sampler makes a sampling decision, that is, it decides whether tracing data is captured for a particular request or not. Instana OpenTelemetry samplers make this decision based on the sampled flag of the trace context. If the value of the sampled flag is not false, the sampler will decide to capture the request.