January 23, 2019 By Hemankita Perabathini 5 min read

Apps may be composed of many microservices—save your admins headaches by enabling proper distributed tracing

In a cloud-native application, multiple microservices are collaborating to deliver the expected functionality. If you have hundreds of services, how do you debug an individual request as it travels through a distributed system? For Java enterprise developers, the Eclipse MicroProfile OpenTracing specification makes it easier.

In our reference implementation, our storefront application is composed of several independent microservice applications, as depicted below:

To assure transactions are delivered reliably and thus keep the database data consistent, the microservices shown above coordinate updates through the open source message queue, RabbitMQ. For the sake of demonstrating the need for microservices-specific tracing, let’s assume RabbitMQ has a problem and has stopped working. If the standard logging was only going to the server hosting the service, finding the source of the problem could require the admin to traipse through logs server-by-server, adding delay to problem resolution:

With distributed tracing, it’s much easier to find where the problem lies and resolve it (e.g., by restarting the server, allocating more disk space, etc.).

With a proper distributed tracing system in place, you have important clues to help you debug the problematic services. Fortunately for Java enterprise developers, you can easily enable it in your MicroProfile application without any explicit code, thanks to MicroProfile OpenTracing.

In our last blog post, David Shi explained how to monitor microservices using MicroProfile Health and Metrics. In today’s entry, I’ll cover the MicroProfile OpenTracing specification we implemented as part of our team’s reference storefront application (GitHub).

This blog series is based on my team’s experience with migrating from Spring Boot-based microservices to MicroProfile, an optimized enterprise Java programming model for a microservices architecture. Both the projects are hosted on GitHub. You can access the Spring Boot version of our reference application here and MicroProfile version of our reference application here.

Monitoring your microservices app in Spring Boot

A commonly used distributed tracing tool for Spring Cloud is Spring Cloud Sleuth. Adding logging traces helps us to debug the transaction flows. Using the traceIDs and spanIDs provided by the Spring Cloud Sleuth, the log statements help identify the individual traces for the existing microservices and allow developers and admins to troubleshoot the cause.

Optionally, you can integrate Zipkin with Spring Cloud Sleuth to add more information to the traces. Simply add Zipkin as a dependency, enable it in your application using the @EnableZipkinServer annotation, and provide the required configurations in the .properties file. It is pretty simple to integrate it and get a nice insight into the transactions and communication happening between different microservices in our application.

public class SpringBootApplication {
    public static void main(String[] args) {
        SpringApplication.run(SpringBootApplication.class, args);

The next section compares the approach to distributed tracing with Spring Cloud Sleuth to the Eclipse MicroProfile implementation, OpenTracing.

Monitoring your microservices app with MicroProfile OpenTracing

MicroProfile OpenTracing enables distributed tracing in our application. It helps us to analyze the transaction flows so that we can easily debug the problematic services and fix them.

In our sample application, we used Zipkin as our distributed tracing system. So, let’s see how we configured WebSphere Liberty to use MicroProfile OpenTracing using Zipkin:

In the server.xml, add opentracingZipkin as a user feature:


Specify the host and port of your Zipkin server in your server.xml:

<opentracingZipkin host="${env.zipkinHost}" port="${env.zipkinPort}"/>

In order to enable custom tracing in your application, we need to implement the OpenTracing Tracer interface. In WebSphere Liberty,  we have the Zipkin server implementation defined as a user feature. To install and download this feature, add the Maven dependency in your pom.xml as shown below:

<!-- Plugin to install opentracing zipkin tracer -->

You can also install this feature manually without using this dependency; see Enabling distributed tracing for details.

Once the MicroProfile OpenTracing mpOpenTracing-1.0 feature is enabled, by default, the distributed tracing is enabled for all the JAX-RS methods in our application. You can also further customize the traces using the @Traced annotation and an ActiveSpan object to retrieve messages.

Now, let’s consider the Catalog service in our reference implementation. This is a sample snippet that gives you an idea of how we defined custom traces in our sample application:


public class CatalogService {
  Tracer tracer;

  @Timeout(value = 2, unit = ChronoUnit.SECONDS)
  @Retry(maxRetries = 2, maxDuration = 2000)
  @Fallback(fallbackMethod = "fallbackInventory")
  @Traced(value = true, operationName = "getCatalog.list")

  public List<Item> getInventory() {
    // Return all items in Inventory

  public List<Item> fallbackInventory() {
    try (ActiveSpan childSpan =
        .buildSpan("Grabbing messages from Messaging System")
        .startActive()) {
          // Return a default fallback list


Defining custom traces is very easy. @Traced  includes an option to disable the default tracing. You can set the value parameter to false to disable it. With this, you can also name your spans using the operationName parameter. You can also define customized traces using a custom tracer object by injecting it with the @Inject annotation.

Now that you are all set to use distributed tracing in your application, what does the result look like? Let’s take a look at the Zipkin traces of an actual logging of a problem in our storefront app. In this example, I’ve intentionally terminated the RabbitMQ service. When this happens, the updated stock will not be passed to the inventory service and thus the MySQL database will not be synchronized. A clear indicator of a problem is the trace depth of “3” instead of the expected “6” from a completed transaction sequence:

If the admin doesn’t have logging indicating the source of the problem (i.e., RabbitMQ is down), it’s not easy to identify the source of the problem because from the user’s point of view, the web interface is (almost) working.

Below is the Zipkin trace details for the sequence above; it includes a log of the HTTP 500 error interrupting the workflow of the application:

Compare this to a normal operational logging of the same transaction:

To run the whole application together, please check out BlueCompute – MicroProfile Implementation.

You can speed problem resolution with proper distributed tracing

In a large organization, your applications may be composed of hundreds of microservices—save your SREs headaches by enabling proper distributed tracing. It will speed problem resolution by simplifying the effort to isolate a failure to a specific microservices application and providing important context surrounding the problem. MicroProfile OpenTracing does (most of) the hard work for you.

For enabling open tracing in this application, I found the Open Liberty guides very useful, specifically the guide Enabling distributed tracing in microservices (20 minutes to read).

This blog excerpts the code and specifications for distributed tracing in our sample application using MicroProfile OpenTracing. You can see all the code available on GitHub. The microservices in our simple storefront application can be run individually using Maven—you can import them and run them as-is locally. You can also run them on IBM Cloud and IBM Cloud Private.

More from Hybrid cloud


3 min read - Migrate from your current JEE app server to Open Liberty in a few easy steps The nine-part series Migrating Java Microservices from Spring Boot to MicroProfile covered the underlying components of Eclipse MicroProfile, such as the Rest Client for creating APIs for microservice applications, Context Dependency Injection for supporting annotation-based wiring, Fault Tolerance for problem recovery, and OpenTracing for diagnosing service issues. This short blog touches on a specific how-to: Simple steps to migrate from your current JEE application server to Open Liberty, an enterprise-grade…

Expanding Data Warehouse Capabilities for the IBM Hybrid Data Management Platform

2 min read - You can now deploy both Flex and Flex Performance instances with the IBM Hybrid Data Management Platform Finding a truly elastic and scalable warehousing solution can make or break your business, especially as today’s data architectures become more and more complex. As companies continue to support new use cases, workload types, and locations, it’s clear that businesses need the freedom and flexibility to choose the right warehouse for the job. That’s why at the end of last year, Db2 Warehouse on…

Modernize Your WebSphere Apps to the Cloud

2 min read - IBM Cloud Application Platform: Combine the best of WebSphere Liberty and IBM Cloud runtimes Many IT shops in industry sectors such as healthcare, finance, and government have a large portfolio of WebSphere Family/Java EE applications that deliver critical functionality. To extend the value of these investments in Java EE apps, tech leaders are weighing the best way to marry today’s advanced cloud services with the capabilities of their existing monolithic architectures. Refactoring these monoliths as microservices applications has emerged as…

Istio and Knative: Extending Kubernetes for a New Developer Experience

6 min read - Istio and Knative are poised to change how application developers use and view Kubernetes Most people already know about Kubernetes as the de facto hosting platform for container-based applications. If you manage a Kubernetes cluster, you probably already know about many of its extensibility points due to the customizations you may have installed. You may have even developed one yourself—such as a custom scheduler—or even gone so far as to extend the Kubernetes resource model by creating your own Custom…

IBM Newsletters

Get our newsletters and topic updates that deliver the latest thought leadership and insights on emerging trends.
Subscribe now More newsletters