Configuring Java sensor

Customizing the application name

The application name identifies the Java application in Instana. The application name can be configured in the following ways: automatic and manual.

Automatic naming

Typically, Instana picks the application name automatically from the following sources:

  • Application .jar file: Within the .jar file, the following standard attributes are available in META-INF/Manifest.MF:

    Implementation-Title=Demo App
    Implementation-Version=1.0.0
     

    By using build tools, you can set both attributes when you build the application. For more information, see the Oracle documentation.

  • Command line: If the name is unavailable, the command line is used to find the name in the same way Java tools display JVMs.

  • System properties: If no name is obtained from the .jar file, Instana inspects various system properties, for example -Dcatalina.home for Tomcat containers.

Manual naming

To customize the application name, use one of the following options:

  • Configure the name during JVM initialization: To explicitly set the application name, specify the -Dcom.instana.agent.jvm.name="My Custom Name" attribute when you start the JVM.

  • Use existing system property: To configure the agent to use an existing custom system property, specify a list of properties in the agent configuration.yaml file. The first existing property names the JVM:

com.instana.plugin.java:
  name-properties:
    - 'acme.app'
    - 'DEPLOYMENT'
 

Customizing the polling interval

You can configure how often Instana polls the JVM to collect data and metrics by using the `poll_rate` parameter in the agent's `configuration.yaml` file.

Configuring poll rate

com.instana.plugin.java:
   poll_rate: 1 # Value is in seconds. Default value is 1 second.
 

Configuring custom JMX metrics

To configure which JMX metrics to monitor, edit the agent configuration.yaml file as shown in the following example:

com.instana.plugin.java:
  jmx:
    # JMX is NOT hot-reloaded and needs to be set before a JVM is discovered.
    # Supported attribute types are Number and Boolean
    # delta calculation is only supported for Number
    - object_name: 'java.lang:type=Compilation'
      metrics:
        - attributes: 'TotalCompilationTime'
          type: 'delta' # delta will report the change to the previous second
    - object_name: 'java.lang:type=ClassLoading'
      metrics:
        - attributes: 'LoadedClassCount'
          type: 'absolute' # absolute will report the value as-is
 

The preceding example specifies the following types of metrics:

  • delta: Used for values that increase over time, such as counters, where the relative change is considered interesting.
  • absolute: Used to show the actual value each time the metric is accessed.

The specified attribute must either be a Number (or one of its subclasses, such as Integer or Double) or a String representing a number.

Configuring JMX metrics with wildcard patterns

The object_name might contain wildcard patterns as specified in the Java ObjectName API documentation.

The following example shows the configuration with wildcard patterns:

com.instana.plugin.java:
  jmx:
    - object_name: 'kafka.*:type=*-metrics,client-id=*,*'
      metrics:
        - attributes: 'incoming-byte-rate'
          type: 'absolute'
        - attributes: 'incoming-byte-total'
          type: 'absolute'
        - attributes: 'outgoing-byte-rate'
          type: 'absolute'
        - attributes: 'outgoing-byte-total'
          type: 'absolute'
 

Viewing custom JMX metrics

On the JVM dashboard, the JMX metrics are displayed under the Custom JMX metrics.

Configuring Java Trace instrumentation

After you install the Instana host agent, the default (non-SDK) Java Trace instrumentation is enabled by default, but the OpenTracing instrumentation and Java Trace SDK are not enabled.

Disabling Java Trace instrumentation

By default, Java trace instrumentation is enabled for all the Java processes. You can disable it for all processes or for a single process.

Disabling for all Java processes

To disable the default Java Trace instrumentation for all processes, configure the agent configuration.yaml file as follows:

com.instana.plugin.javatrace:
   instrumentation:
     enabled: false
 
Note: Restart the Java application to make sure that the instrumentation is disabled.

To explicitly enable Java Trace instrumentation, set enabled: false to true.

Disabling for a single Java process

Java trace instrumentation is enabled for all the Java processes by default.

To disable trace instrumentation for a Java process, use one of the following options:

  • Add the -Dinstana.java.tracer.enabled=false system property as a Java virtual machine (JVM) argument.
  • Add the **INSTANA_JAVA_TRACER_ENABLED=false **system environment variable.
Note: The system property is prioritized over the system environment variable if both are provided.

Enabling OpenTracing instrumentation

To enable the OpenTracing instrumentation of Java Trace, configure the agent configuration.yaml file as follows:

com.instana.plugin.javatrace:
   instrumentation:
     opentracing: true
 

To disable the OpenTracing instrumentation, set opentracing: false to false.

Enabling Java Trace SDK

To enable the Java Trace SDK, configure the agent configuration.yaml file by providing a list of packages that need to be scanned for SDK annotation as follows. By default, no packages are scanned for SDK annotations.

com.instana.plugin.javatrace:
   instrumentation:
     sdk:
       packages:
         - 'com.instana.backend'
         - 'com.instana.frontend'
 

To disable the Java Trace SDK, comment the entire packages section.

Capturing intermediate spans

Instana can capture intermediate spans for Java applications in Kafka Streams and WebMethods Integration Server. However, this feature is disabled by default.

Note: Enabling intermediate spans results in a substantial increase in the number of spans, multiplying the volume of trace data and potentially impacting the application performance.

To enable this feature, see the following sections:

Enabling intermediate spans for Kafka Streams

In Kafka Streams, every stream operator is captured as an intermediate span. This feature impacts the performance of the application if you have large data inflows. Do not use this feature in production environments.

To enable capturing of intermediate spans for Kafka Streams, add the following lines to your agent configuration.yaml file:

com.instana.plugin.javatrace:
   instrumentation:
    plugins:
     Kafka011StreamIntermediate: true
 

Enabling internal service calls for WebMethods Integration Server

You can enable the feature of capturing every service call in WebMethods Integration Server as an intermediate span. But if the server handles heavy workloads, this feature can impact the performance of your application.

To enable capturing internal service calls for WebMethods Integration Server, add the following lines to your agent configuration YAML file:

com.instana.plugin.javatrace:
   instrumentation:
     plugins:
       WebmethodsIntermediate: true
 

Disabling stack trace capture

Java Tracer captures stack traces for exit and intermediate spans. However, stack traces for entry spans are disabled by default. If an application has a deeper stack or a request results in several spans, capturing stack traces might impact application performance. If you are facing performance issues, you can disable stack trace to optimize performance.

To disable capturing of stack trace, start the JVM with the following property:

-Dinstana.tracing.stack.exit=-1 -Dinstana.tracing.stack.log=-1
 

The following example displays how to disable capturing of stack trace:

$ java -Dinstana.tracing.stack.exit=-1 -Dinstana.tracing.stack.log=-1 -jar application.jar
 

Configuring span correlation

By default, when consumers retrieve messages, a single span is generated to represent the entire receive operation, grouping all messages under one span. To enhance trace granularity, a configuration option is available that enables correlation on a per-message basis.

Configuring Kafka span correlation for enhanced observability

When Kafka consumers retrieve messages, Instana typically generates a single span to represent the entire polling operation. This means all messages fetched during one poll are grouped under a single POLL span, which can limit visibility into how each individual message is processed.

To improve trace granularity, Instana provides a configuration option that enables correlation for each individual send and receive operation—even when messages are consumed in batches. This allows the creation of separate spans for every message, providing deeper insights into message-level processing.

Note: Kafka Span correlation is disabled by default (values set to false).

To enable Kafka span correlation, use one of the following configurations:

  • Agent configuration: Configure the agent configuration.yaml file.
    com.instana.tracing:  
      kafka:  
        correlateEverySendReceive: true # possible values: true or false
    Note:

    Restart the agent or application after editing the agent configuration.yaml file.

  • Environment variable configuration: Set the environment variable as shown in the following example:
    INSTANA_KAFKA_CORRELATE_SEND_RECEIVE=true # possible values: true or false
  • System property configuration: Set the following property when you start the JVM:
    -Dcom.instana.kafka.correlate.send.receive=true # possible values: true or false 

The following images illustrate how span correlation behaves with and without the configuration enabled.

Figure 1. Span correlation without configuration enabled
Span correlation without configuration enabled

Figure 2. Span correlation with configuration enabled
Span correlation with configuration enabled

To completely disable Kafka span correlation in Instana, use one of the following configurations:

  • Agent configuration:
    com.instana.tracing: 
      kafka: 
        correlateEverySendReceive: false 
  • Environment variable configuration: Set the environment variable.
    INSTANA_KAFKA_CORRELATE_SEND_RECEIVE=false
  • System property configuration: Provide the option -Dcom.instana.kafka.correlate.send.receive=false when you start the JVM.

Configuring trace correlation

Instana trace correlation is disabled by default for technologies, such as IBM Db2, WebLogic T3, IBM MQ, and Kafka, which use Java Tracer for tracing. To enable trace correlation for these technologies, see the following sections:

Configuring IBM Db2 trace correlation

To configure IBM Db2 trace correlation, use one of the following configuration options:

  • Agent configuration: Configure the agent configuration.yaml file as follows:

      com.instana.tracing:
        db2:
          trace-correlation: true
          db2CorrelationMode: dont-override
          db2CorrelationDelimiter: '|'
          db2CorrelationField: ClientHostname
     
  • Environmental variable configuration: Set the following environmental variables:

     INSTANA_DB2_TRACE_CORRELATION=true
     INSTANA_DB2_TRACE_CORRELATION_MODE=dont-override
     INSTANA_DB2_TRACE_CORRELATION_DELIMITER='|'
     INSTANA_DB2_TRACE_CORRELATION_FIELD=ClientHostname
     
  • System property configuration: Set the following system properties in the instrumented JVM:

     -Djdbc.db2.correlation.enable=true
     -Djdbc.db2.correlation.mod=dont-override
     -Djdbc.db2.correlation.delimiter=|
     -Djdbc.db2.correlation.field=ClientHostname
     
Note: By default, correlation (propagating TraceId and SpanId) is disabled. Propagation happens by using client information properties of Db2. The correlation field sets the field name as ClientHostname by default. The default delimiter is |. If the correlation mode is set to override (default), the correlation data overwrites any existing value. However, if it is not set to override, the correlation data is appended to the existing value.

Configuring WebLogic T3 trace correlation

Enterprise Java Bean (EJB) calls support WebLogic T3 correlation.

To configure WebLogics T3 trace correlation, use one of the following configuration options:

  • Agent configuration: Configure the agent configuration.yaml file as follows:

      # General Tracing Settings
      com.instana.tracing:
        weblogic:
          trace-correlation: true
     
  • Environmental variable configuration: Set the following environment variable:

    INSTANA_WEBLOGIC_EJB_TRACE_CORRELATION=true
     
  • System property configuration: Set the following system property in the instrumented JVM:

    -Dcom.instana.weblogic.ejb.trace.correlation=true
     

    This configuration works only if the T3 communication happens between WebLogic servers. However, if the WebLogic client is used outside the WebLogic server, correlation is not possible.

Enabling IBM MQ trace correlation

In some IBM MQ consumer clients, the presence of extra header data in IBM MQ messages can cause message processing errors and message rejection. Therefore, Instana Trace header correlation is not enabled by default for IBM MQ Java applications.

You must test IBM MQ trace correlation in a nonproduction environment before you enable it in a production environment. If you encounter errors on the IBM MQ client applications after you enable trace correlation, troubleshoot as follows:

  • If you can alter the application, update the application to ignore the additional IBM MQ header data that is added by Instana. For assistance to make IBM MQ client changes, contact IBM MQ support.
  • If you cannot alter the application, do not enable the IBM MQ trace correlation support.

To enable trace correlation for IBM MQ, use one of the following configuration options:

  • Agent configuration: Edit the agent configuration.yaml file as shown in the following example:

    # General Tracing Settings
    com.instana.tracing:
      ibm-mq:
        trace-correlation: true
        ibmmq-optin-queues-regexp: '^QueueName.*'
     

    Set the ibmmq-optin-queues-regexp regular expression value to filter the queue names for which you want to enable the trace correlation. The regular expression value is optional and if it is missing then the trace correlation applies to all queues.

    This configuration is picked up in any of the following scenarios:

    • When the JVMs starts
    • When the Java Tracer is updated
    • After the Instana agent is shut down for approximately 15 minutes

    It is not hot-reloaded at run time. A quick agent restart might not pick up this configuration change because this feature is optimized. A configuration change needs a full reinstrumentation.

  • System property configuration: Alternately, specify a system property on the application JVMs that contain IBM MQ clients as shown in the following example:

    -Dinstana.ibm.mq.trace.correlation.enable=true -Dcom.instana.ibm.mq.trace.optin.queues="^QueueName.*"
     

The IBM MQ sensor also provides a trace capability by using user exits (server-side tracing for IBM MQ). For more information about the IBM MQ sensor, see Monitoring IBM MQ.

Configuring Kafka trace correlation headers

To enable the header format for Kafka trace correlation headers that are used by the Java Tracer, use one of the following configuration options:

  • Agent configuration: Configure the agent configuration.yaml file. For more information, see the Instana host agent.

  • Environment variable configuration: Set the environment variable as shown in the following example:

    INSTANA_KAFKA_HEADER_FORMAT= both # possible values: binary, both, string
     

    Valid values are binary, string, or both.

  • System property configuration: Set the following property when you start the JVM:

    -Dcom.instana.kafka.trace.header.format=binary # possible values: binary, both, string
     

To disable Kafka Trace correlation entirely, use one of the following configuration options:

  • Agent configuration: See Instana host agent.
  • Environment variable configuration: Set the environment variable INSTANA_KAFKA_TRACE_CORRELATION=false, but the use of this option is not suggested.
  • System property configuration: Provide the option -Dcom.instana.kafka.trace.correlation=false when you start the JVM.

For more information, see Kafka Header Migration.

Configuring span batching

Span batching is a feature that merges multiple spans and sends them to the agent in batches, rather than individually. When span batching is enabled, spans with the same name, type, and duration (less than 10 ms) are aggregated. Typically, high-frequency and short spans, such as database calls, are batched together.

The following image shows the implementation of span batching. The dashboard highlights 5 sub calls, of which 3 are batched and listed in the Calls section.

Figure 3. Span batching
Span batching
Note: Span batching is enabled by default. To explicitly enable span batching, set the INSTANA_SPANBATCHING_ENABLED parameter to true.

Span batching is enabled automatically for the following applications: Aerospike, Amazon s3, Cassandra, Data Grid, DynamoDB, Elasticsearch, FTP, Google Cloud Bigtable, Google Cloud storage, HBase, Hazelcast, IMS, JavaMail, Kinesis, LDAP, Memcached, MongoDB, Neo4j, and Redis.

For Data Grid and JDBC, specific span merge strategies are followed. For more information, see Span batching strategies.

Disabling span batching

If you disable span batching, the number of individual spans sent increases, resulting in a higher data volume.

To disable span batching, use any one of the following configuration options:

  • Agent configuration: Configure the agent configuration file as follows:

      com.instana.plugin.javatrace:
        instrumentation:
          spanBatchingEnabled: false
     
  • Environmental variable configuration: Set the environmental variable as follows:

    INSTANA_SPANBATCHING_ENABLED=false
     
  • System property configuration: Set the system property in the instrumented JVM as follows:

    -DspanBatchingEnabled=false
     

Span batching strategies for Data Grid and JDBC spans

Applications, such as Data Grid and JDBC, implement the following custom batching strategies for span batching:

  • Data Grid: Span are merged only if both datagrid.operation and datagrid.cacheName have the same value in sequential spans. Span batching is enabled only if this condition is met.

  • JDBC: The following types of span batching strategies are supported:

    • Default batching (enabled by default)

      Spans are merged regardless of the value of JDBC span attributes.

    • Custom batching (optional)

      Spans are merged only if the value of JDBC.statement is identical in sequential spans.

Enabling custom batching for JDBC spans

The size of the ingested data increases if you enable custom batching for JDBC spans.

To enable custom batching for JDBC spans, use any of the following configuration options:

  • Agent configuration: Configure the agent configuration file as follows:

      com.instana.plugin.javatrace:
        instrumentation:
          customJDBCSpanBatchingEnabled: true
     
  • Environmental variable configuration: Set the environmental variables as follows:

    INSTANA_CUSTOM_JDBC_SPANBATCHING_ENABLED=true
     
  • System property configuration: Set the system property in the instrumented JVM as follows:

    -DcustomJDBCSpanBatchingEnabled=true
     

Configuring logging levels

You can enable and disable the logging levels in Instana. Instana supports the following logging levels:

  • OFF
  • ERROR
  • WARN

When a logging level is set, all higher priority logging levels are included in tracing. For example, if ERROR is configured, the logging levels ERROR and FATAL are traced.

Enabling customer-defined logging level

To configure the logging level, use one of the following configuration options:

  • System property configuration: Set instana_log_level_capture to ERROR or WARN.
  • Environment variable configuration: Set INSTANA_LOG_LEVEL_CAPTURE to ERROR or WARN.

If you provide an invalid logging level (for example, INFO or TRACE), the following warning is logged in the agent logs and the default logging level WARN is used.

The value 'INFO' for 'INSTANA_LOG_LEVEL_CAPTURE' is not valid. Valid values are 'WARN', 'ERROR' and 'OFF'.
 

Disabling logging plug-ins

To disable logging for all logging plug-ins, you can use one of the following configuration options:

  • Agent configuration: Configure Java tracing to disable all logging plug-ins in the agent configuration.yaml file:

        com.instana.plugin.javatrace:
          instrumentation:
            plugins:
              Logging: false
     
  • System property configuration: Set instana_log_level_capture="OFF".

  • Environment variable configuration: Set INSTANA_LOG_LEVEL_CAPTURE="OFF".

Configuring Java Tracer for JVMs running incompatible agents

The Instana agent does not automatically monitor JVMs that also run any of the following agents:

  • AppDynamics
  • DataDog
  • Dynatrace
  • Glow Root
  • Kanela, the Kamon Instrumentation Agent
  • ManageEngine APM Insight Java Agent
  • New Relic
  • Oracle Java Flight Recorder
  • OpenTelemetry Instrumentation for Java
  • Riverbed AppInternals
  • Spring Loaded
  • Wily Introscope, also known as CA Application Performance Management and Broadcom DX Application Performance Management

To monitor JVMs that run any of these agents, configure the agent configuration.yaml file as follows:

com.instana.plugin.javatrace:
  trace-jvms-with-problematic-agents: true
 

Ignoring classes

To disable instrumentation of a particular class or classes in a package, you can use one of the following configuration options:

  • Agent configuration: Configure Java Trace sensor to ignore instrumentation of a class or classes in a package in the agent configuration.yaml file as follows:

        com.instana.plugin.javatrace:
          instrumentation:
             ignoredClass: "^ClassName or PackageName"
     

    For example, use the following configuration to disable instrumentation of all classes in the org.apache.derby package:

           com.instana.plugin.javatrace:
             instrumentation:
                ignoredClass: org.apache.derby.
     
    Note: Restart the agent or application after editing the agent configuration.yaml file.
  • System property configuration: Set ignoredClass="^ClassName or PackageName" as shown in the following example:

    -DignoredClass=org.postgresql
     
  • Environment variable configuration: Set IGNORED_CLASS_NAME="^ClassName or PackageName".

Ignoring endpoints

This feature helps to reduce unnecessary data ingestion by excluding specific traces or calls. This filtering method refines your trace collection. To exclude traces, define this parameter in any one of the following Java Tracer configuration settings:

This feature is available for the following technologies:

  • DynamoDB
  • Redis
  • Kafka

Considerations for ignoring endpoints

Before you configure the ignore endpoints feature, review the following rules for filtering spans:

  • When a trace is ignored, all subsequent downstream traces are also ignored.
  • Use * to ignore all endpoints, methods, or commands.
  • Endpoint values (such as Kafka topic names or Redis commands) remain consistent across services.
  • Method names might vary depending on the programming language and technology. Refer to the Instana UI to determine the correct method and endpoint of your service.

The following Instana UI screenshot provides visual reference for identifying the correct method and endpoint for configuration:

Figure 4. Methods and endpoints in the Instana UI
Methods and endpoints

The ignore endpoints feature works with only exact method names. Ensure that you use the correct method name to be filtered.

Agent configuration

You can globally ignore endpoints for all your services by configuring them through the agent.

To configure the endpoints to be ignored, specify the endpoints that must be excluded from monitoring in the com.instana.tracing.ignore-endpoints section in your agent configuration file as shown in the following example:

com.instana.tracing:
  ignore-endpoints:

    # Filtering by Method Name
    redis:
      - 'get'
      - 'type'
    dynamodb:
      - 'query'

    # Filtering by Method Name and Endpoint for Kafka
    kafka:
      - methods: ["consume"]
        endpoints: ["topic1", "topic2"]  # Exclude consume calls for topic1 and topic2
      - methods: ["consume", "publish"]
        endpoints: ["topic3"]  # Exclude both consume and send calls for topic3
      - methods: ["*"]
        endpoints: ["topic4"]  # Exclude all methods for topic4
      - methods: ["consume"]
        endpoints: ["*"]  # Exclude consume method for all topics

In the preceding example, the following traces are ignored for the listed filtering options:

  • Filtering by method (Redis and DynamoDB)

    • GET and TYPE commands in Redis
    • QUERY command in DynamoDB
  • Filtering by method and endpoint (Kafka only)

    • CONSUME method for topic1 and topic2 in Kafka and all downstream traces
    • CONSUME and PUBLISH methods for topic3 in Kafka and all downstream traces
    • All methods (*) for topic4 in Kafka and all downstream traces
    • CONSUME method for all topics (*) and all downstream traces

Method options

For Kafka methods, you can use the following options:

  • Consumer: consume and poll
  • Producer: publish and send

The number of endpoints for Kafka are not limited. Therefore, a large number of endpoints can potentially impact performance. Hence, carefully consider the usage of endpoints size for Kafka. Redis and DynamoDB endpoints list size is limited to 10.

Kafka ignore endpoints has an important characteristic that can merge configuration based on the usage of *. The following example illustrates this characteristic:

  ignore-endpoints:
    kafka:
      - methods: [ "consume", "publish" ]
        endpoints: [ "topic1", "topic2" ]
      - methods: [ "consume" ]
        endpoints: [ "topic3" ]
      - methods: [ "*" ] # Applied to all methods
        endpoints: [ "topic4" ]
      - methods: [ "consume", "publish" ]
        endpoints: [ "*" ] # Applied to all topics

In the preceding example, the following traces are ignored for the listed filtering options:

  • All topics are ignored because of the last entry where * is applied for consume and publish methods. In this case, all the other entries are overridden.
  • None of the previous entries take effect.

Environment variables

You can configure ignored endpoints by using the INSTANA_IGNORE_ENDPOINTS environment variable to filter endpoints per service.

With this environment variable, you can exclude traces based on methods and endpoints.

INSTANA_IGNORE_ENDPOINTS=redis:get,type;dynamodb:query,scan;kafka:publish,consume

This configuration filters the following traces:

  • GET and TYPE commands in Redis
  • QUERY and SCAN commands in DynamoDB
  • PUBLISH and CONSUME methods in Kafka for all the endpoints

For Kafka, you can apply the environment variables for ignoring endpoints, but only one endpoint and one method can be specified at a time.

INSTANA_IGNORE_ENDPOINTS=kafka:topic1,topic2,topic3,topic4,topic5,topic6,topic7,topic8,topic9,topic10,topic11

For Kafka, unlike Redis and DynamoDB, the list of endpoints is not limited to 10. Therefore, you must carefully consider the size of the list because it can impact the performance.

By using method options for Kafka, configure the environment variable as follows:

INSTANA_IGNORE_ENDPOINTS=kafka:consume,publish
OR
INSTANA_IGNORE_ENDPOINTS=kafka:poll,send
OR
INSTANA_IGNORE_ENDPOINTS=kafka:*

System properties

You can also use Java system property to configure ignore endpoints.. The following example shows configuring ignore endpoints by using system properties:

java -Dinstana.ignore-endpoints="redis:get,type;dynamodb:query,scan;kafka:publish,consume" -jar application-jar

For Kafka, you can apply the the system property for ignoring endpoints, but only one method and one endpoint can be specified at a time.

java -Dinstana.ignore-endpoints="kafka:topic1,topic2,topic3,topic4,topic5,topic6,topic7,topic8,topic9,topic10,topic11" -jar application-jar

For Kafka, unlike Redis and DynamoDB, the list of endpoints is not limited to 10. Therefore, you must carefully consider the size of the list because it can impact the performance.

By using method options for Kafka, configure the system property as follows:

java -Dinstana.ignore-endpoints="kafka:consume,publish" -jar application-jar
OR
java -Dinstana.ignore-endpoints="kafka:poll,send" -jar application-jar
OR
java -Dinstana.ignore-endpoints="kafka:*" -jar application-jar