Monitoring Java virtual machine (JVM) by using OpenTelemetry

Instana provides a holistic view of JVM metrics that are collected by using OpenTelemetry. After you integrate OpenTelemetry with Instana, you can view these metrics in the Instana UI.

Instana supports standard OpenTelemetry Java automatic or manual instrumentation. For more information, see OpenTelemetry official documents.

Prequisites

Ensure that the following prerequisite are met:

Vendor-provided solutions are available for OpenTelemetry. For example, Spring provides its own built-in OpenTelemetry solution.

Instrumentation

OpenTelemetry is a widely used open source project for collecting and managing telemetry data from applications. One of the key aspects of OpenTelemetry is instrumentation, which involves adding code to your application to collect metrics, logs, and traces.

To enable instrumentation, you can use one of the following methods:

Auto-instrumentation

Auto instrumentation is a feature provided by OpenTelemetry to automatically instrument your application without writing custom code. Auto instrumentation helps you to collect telemetry data without modifying the source code of the application.

Two versions of OpenTelemetry Java agents are available; v1.x and v2.x. These versions are supported to accommodate major changes and improvements in the OpenTelemetry specification. OpenTelemetry introduced new semantic conventions for HTTP, which affected metric naming and structure. To align with these changes and provide accurate, standardized data, a new major version (2.x) was necessary. Version 1.x continues to support older conventions for backward compatibility. To enable auto-instrumentation, complete the following steps:

  1. Download the OpenTelemetry Java agent by running the following command:

    wget https://github.com/open-telemetry/opentelemetry-java-instrumentation/releases/latest/download/opentelemetry-javaagent.jar

  2. Run the application with the OpenTelemetry Java agent as shown in the following example:

    export OTEL_SERVICE_NAME=my-service
export OTEL_TRACES_EXPORTER=otlp
export OTEL_METRICS_EXPORTER=otlp 
export OTEL_LOGS_EXPORTER=otlp 
export OTEL_EXPORTER_OTLP_ENDPOINT="http://localhost:4317"
export OTEL_EXPORTER_OTLP_PROTOCOL=grpc
export OTEL_RESOURCE_ATTRIBUTES="service.instance.id=petclinic,INSTANA_PLUGIN=jvm"
export OTEL_EXPORTER_OTLP_HEADERS="x-instana-key=instanalocal"
java -javaagent:./opentelemetry-javaagent.jar -jar target/*.jar --server.port=28080

Manual instrumentation

Manual instrumentation involves writing custom code to instrument specific parts of your application. Manual instrumentation requires you to manually configure the source code of the application to configure OpenTelemetry SDK to receive, process and export the telemetry data.

Sending telemetry data to Instana

Instana receives OpenTelemetry data through various methods that adhere to open standards, helping to ensure seamless integration with the OpenTelemetry community and Instana product. The implementation is actively maintained by both parties, which guarantees compatibility and reliability.

Sending telemetry data to Instana agent

The Instana agent can directly receive OpenTelemetry traces, metrics, and logs in the OTLP format. Then, the Instana agent forwards the data to the Instana backend. For more details, see Send data to Instana Agent.

Sending telemetry data to Instana backend

Telemetry data can be sent in the OTLP format directly from applications or systems to the Instana backend, which is also referred to as agent-less communication. For more details, see Send data to Instana Backend.

Viewing JVM metrics

To view JVM metrics, complete the following steps:

  1. From the navigation menu in the Instana UI, click Infrastructure > Analyze infrastructure.
  2. Select OTEL JVM from the list of types of the entities.
  3. Click the instance.

The OTEL JVM dashboard displays all the collected metrics for the instance. The following image displays a typical OTEL JVM instance dashboard:

OpenTelemetry jvm

Performance metrics

Metrics Description Source
Total Heap Usage The amount of heap memory that is used. Derived from the jvm.memory.used metric.
Thread States The number of platform threads in the runtime. Derived from the jvm.thread.count metric.
Heap Memory The amount of heap memory that is used in absolute terms and as a percentage of the total heap. Derived from the jvm.memory.used metric.
Memory Pools The measure of current and maximum memory that is used for different memory pools. Derived from the jvm.memory.max metric.
Garbage Collected Heap Memory The measure of memory that is used as measured after the most recent garbage collection event on this memory pool. Derived from the jvm.memory.used_after_last_gc metric.
Limit of Memory Pools The measure of maximum obtainable memory. Derived from the jvm.memory.limit metric.
Init of Memory Pools The measure of initial memory that is requested. Derived from the process.runtime.jvm.memory.init metric.
Initial values of Memory Pools The measure of initial values of memory that is used for different memory pools. Derived from the jvm.memory.heap metric.
Memory used after GC The measure of memory that is used as measured after the most recent garbage collection event on this memory pool. Derived from the process.runtime.jvm.memory.usage_after_last_gc metric.
Memory used after GC The measure of memory that is used as measured after the most recent garbage collection event on this memory pool. Derived from the jvm.memory.used_after_last_gc metric.
Number of executing threads The number of executing platform threads. Derived from the process.runtime.jvm.threads.count metric.
Number of peak threads The number of peak threads count in the JVM. Derived from the jvm.threads.peak metric.
Number of classes loaded The number of classes that are loaded since the JVM started. Derived from jvm.classes.loaded metric.
Number of classes loaded The number of classes that are loaded since the JVM started. Derived from the jvm.class.loaded metric.
Number of classes unloaded The number of classes that are unloaded since the JVM started. Derived from the jvm.classes.unloaded metric.
Number of classes unloaded The number of classes that are unloaded since the JVM started. Derived from the jvm.class.unloaded metric.
CPUs count The number of processors that are available in the system. Derived from the system.cpu.count metric.
CPUs count The number of processors that are available to the JVM. Derived from the jvm.cpu.count metric.

Troubleshooting

OpenTelemetry JVM instance is not listed in the dashboard

If OpenTelemetry JVM instance is not listed in the dashboard, complete the following steps:

  1. Check Instana Host Agent logs to see whether any messages are related to this OTEL JVM process. OpenTelemetry JVM process was properly instrumented either automatically or manually. Check whether instrumentation steps followed as per recommendation or not.
  2. Check whether environment variables are set properly.
  3. Check whether OpenTelemetry endpoints are defined properly (in configuration files, Helm charts, and so on) to communicate with the agent.

References