Integrating with OpenTelemetry by using the OpenTelemetry Collector

OpenTelemetry Collector

The OpenTelemetry official distributions include OpenTelemetry Collector (otelcol) and OpenTelemetry Collector Contrib (otelcol-contrib), which are available as part of the OpenTelemetry Collector releases repository.

The OpenTelemetry Collector is the core component of the OpenTelemetry ecosystem, offering vendor-independent function for telemetry data collection, processing, and export. To explore its capabilities, download the latest executable binary files from GitHub releases.

The OpenTelemetry Collector Contrib extends the core collector with extra components and community-contributed enhancements, providing specialized functionalities for advanced use cases. While some of the components in this repository are part of the otelcol distribution, but most of the components here are only available as part of the otelcol-contrib distribution. You can build your own custom distributions with the OpenTelemetry Collector builder by using the components that you need from the core repository, the contrib repository, and third-party or internal repositories. To use these enhancements for your telemetry data collection needs, download the latest executable binary files from OpenTelemetry Collector releases.

Moreover, OpenTelemetry Collector releases are available as container image binary files that ensure ease of deployment across various environments.

For more information about installation instructions for the OpenTelemetry Collector, see Install the Collector.

Setting resource attributes with the OpenTelemetry Collector

You can set resource attributes with the OpenTelemetry Collector. For more information, see Using the OpenTelemetry Collector to set resource attributes.

Scenarios for using the OpenTelemetry Collector with Instana

OpenTelemetry data can be directly transmitted to either an Instana agent or the Instana backend, given their compatibility with both OTLP/gRPC and OTLP/HTTP protocols.

The OpenTelemetry Collector comprises the following essential components:

• Receiver: Collects telemetry data from various sources and converts it into a standardized format for processing.
• Processor: Modifies and enhances telemetry data, allowing for tasks such as filtering, aggregation, and attribute addition.
• Exporter: Sends processed telemetry data to external destinations in the required format, facilitating integration with observability platforms and other tools.

The OpenTelemetry Collector offers significant enhancements in data processing, manipulation, and debugging capabilities, while also enabling the configuration of multiple data export destinations. Therefore, using the OTLP endpoint of an Instana agent or backend to configure either the otlp exporter or otlphttp exporter within the OpenTelemetry Collector ensures that data handling is efficient and streamlined.

While the OpenTelemetry Collector Contrib includes a specific Instana exporter, using a standard OTLP exporter is preferred due to its adherence to standards and its comprehensive support for all OpenTelemetry data signals, including traces, metrics, and logs.

The following examples show how the OpenTelemetry Collector configuration enhances the Telemetry data process:

• The batch processor can be used to improve data process efficiency.
• The spanmetrics processor can be used to extract golden signal metrics from spans.

For the batch processor, see the following sample configuration that is based on performance considerations for most scenarios:

processors:
  batch:
    send_batch_size: 5000
    send_batch_max_size: 10000
...
service:
  pipelines:
    traces:
      receivers: [...]
      processors: [..., batch]
      exporters: [...]
    metrics:
      receivers: [...]
      processors: [..., batch]
      exporters: [...]

For the spanmetrics processor, see the following sample configuration.

Most of metrics that are derived from Instana-related services are based on spans. In most cases, incorporating the spanmetrics processor into the pipeline for exporting metrics to Instana is unnecessary.

processors:
  spanmetrics:
    metrics_exporter: otlp/1
...
pipelines:
  traces:
    receivers: [...]
    processors: [...]
    exporters: [...]
  metrics:
    receivers: [...]
    processors: [...]
    exporters: [..., otlp/1]

The following example shows a typical configuration for debugging data by using the OpenTelemetry Collector:

exporters:
  logging:
    verbosity: detailed
...
pipelines:
  traces:
    receivers: [...]
    processors: [...]
    exporters: [..., logging]
  metrics:
    receivers: [...]
    processors: [...]
    exporters: [..., logging]

com.instana.plugin.opentelemetry:
  grpc:
    enabled: true
  http:
    enabled: true

exporters:
  otlp:
    endpoint: "localhost:4317"
    tls:
      insecure: true
...
service:
  pipelines:
    traces:
      receivers: [...]
      processors: [...]
      exporters: [..., otlp]
    metrics:
      receivers: [...]
      processors: [...]
      exporters: [..., otlp]
    logs:
      receivers: [...]
      processors: [...]
      exporters: [..., otlp]

exporters:
  otlphttp:
    endpoint: "localhost:4318"
    tls:
      insecure: true
...
service:
  pipelines:
    traces:
      receivers: [...]
      processors: [...]
      exporters: [..., otlphttp]
    metrics:
      receivers: [...]
      processors: [...]
      exporters: [..., otlphttp]
    logs:
      receivers: [...]
      processors: [...]
      exporters: [..., otlphttp]

exporters:
  otlp:
    endpoint: otlp-blue-saas.instana.io:4317
    headers:
      x-instana-key: xxxxxxx
      x-instana-host: xxxx

processors:
  resource:
    attributes:
    - key: host.id
      action: upsert
      value: "myhost1.ibm.com"

otel-contrib --config configuration.yaml