Enabling distributed tracing

Configure the environment to enable automatic distributed tracing in JAX-RS applications. You can enable the mpOpenTracing-1.3 feature together with a user-provided opentracing.io.Tracer implementation to allow JAX-RS applications to automatically create, propagate, and deliver distributed tracing information.

Stabilized features: The MicroProfile OpenTracing 1.3 feature is stabilized. The strategic alternative is mpTelemetry-1.0, a MicroProfile Telemetry feature. For more information, see the MicroProfile OpenTracing to MicroProfile Telemetry 1.0 migration document.

Starting with 19.0.0.12, the mpOpenTracing-1.3 feature supports Jaeger as a distributed tracing backend. No user feature is required in the server configuration. For more information, see the Open Liberty blog.

About this task

Distributed tracing: Use distributed tracing to trace the flow of a request across service boundaries, which is important in a microservices environment, where a request typically flows through multiple services. When distributed tracing is active, each service creates trace records with a correlation ID. The correlation ID that is used might have been propagated from an upstream service. A common companion to distributed trace logging is a storage service for the distributed trace records, as shown in these examples. Use the storage service to view the trace records that are associated with particular request flows across services.

Opentracing.io: The opentracing.io project provides a standard API for instrumenting services for distributed tracing. If you instrument every service for distributed tracing with the opentracing.io API, you can configure the service at deployment time. Make the service use a common system implementation to format log records and propagate cross service correlation IDs if an implementation library exists for the language of the service. The common implementation propagates correlation IDs so that they are understandable to all services. The implementation also formats log records so that the server can understand them for distributed trace record storage.

Procedure

Provide a user feature. Create and install a user feature with an io.opentracing.Tracer implementation in the ${wlp.user.dir}/extension directory.
1. Access sample source code for a feature that provides a Zipkin-specific opentracing.io API implementation at Zipkin Opentracing tracer implementation for use with Liberty. Access a built version of the user feature that provides a Zipkin-specific opentracing.io API implementation in the Maven repository.
2. Make the user feature provide an OSGI service that implements the com.ibm.ws.opentracing.tracer.OpentracingTracerFactory interface:
```
package com.ibm.ws.opentracing.tracer;

import io.opentracing.Tracer;

public interface OpentracingTracerFactory {

    Tracer newInstance(String serviceName);

}
```
1. Set the newInstance method of the OpentracingTracerFactory implementation to return an instance of an io.opentracing.Tracer object. Ensure that the Tracer object that the program returns is the correct implementation for the environment where the services run.
  In the example in Zipkin Opentracing tracer implementation for use with Liberty, the Tracer is an instance that communicates in an environment where a Zipkin server is available.
2. If the environment does not use a Zipkin server, change the implementation to provide a Tracer object that communicates with the distributed tracing server in the environment.
  You produce two artifacts when you create a user feature:
  - A .jar file with the OpentracingTracerFactory implementation and the Tracer implementation
  - An mf file with the description of the user feature
  In the example at Zipkin Opentracing tracer implementation for use with Liberty, two files are created at the following locations:
  
  For mpOpenTracing-1.0:
```
target/extension/lib/com.ibm.ws.opentracing.zipkintracer-0.30.jar
target/extension/lib/features/opentracingZipkin-0.30.mf
```
  For mpOpenTracing-1.1, mpOpenTracing-1.2, or mpOpenTracing-1.3:
```
target/extension/lib/com.ibm.ws.opentracing.zipkintracer-0.31.jar
target/extension/lib/features/opentracingZipkin-0.31.mf
```
3. Copy the .jar file with the implementation to the ${wlp.user.dir}/extension/lib directory for all Liberty servers in the environment.
4. Copy the .mf file with the feature description to the ${wlp.user.dir}/extension/lib/features directory for all Liberty servers in the environment.
Configure the server. Enable the user feature in the server.xml files in the server environments.
1. Enable the feature in each of the Liberty servers in the environment.
  The .mf file that you created contains a property that is named IBM-ShortName. The value of the IBM-ShortName property is the feature name that is enabled in the server.xml file of each of the Liberty servers. In the example at Zipkin Opentracing tracer implementation for use with Liberty, the value of the IBM-ShortName property is opentracingZipkin-0.30 or opentracingZipkin-0.31 in mpOpenTracing-1.1, mpOpenTracing-1.2, or mpOpenTracing-1.3. Add the following code to each server.xml file to enable this feature:
  For mpOpenTracing-1.0:
```
<featureManager>
	<feature>usr:opentracingZipkin-0.30</feature>
</featureManager>
```
  For mpOpenTracing-1.1, mpOpenTracing-1.2, or mpOpenTracing-1.3:
```
<featureManager>
	<feature>usr:opentracingZipkin-0.31</feature>
</featureManager>
```
  The sample at Zipkin Opentracing tracer implementation for use with Liberty accepts configuration parameters from the server.xml file. If you cannot reach the Zipkin server in the environment with the default service name zipkin and the default port 9411, configure the host and port by adding an opentracingZipkin configuration element to each server:
```
<opentracingZipkin host="Zipkin Host Name" port="Zipkin Port Number"/>
```