How-tos

Distributed Tracing in a Spring Application using Spring Sleuth

Share this post:

In this post, we’ll create a few Spring applications demonstrating Sleuth’s distributed tracing functionality, using Zipkin for visualization.  The first application will be a Zipkin service; the other apps we create will send tracing data here.  The next application will be a delay service; it will take an integer parameter and wait for the appropriate amount of time before responding.  This app will allow us to simulate any number of other services and see how slow responses are handled.  The last application will call the delay service and pass the time parameter to it.  This network of applications will allow us to build some nested spans and see the power of Sleuth and Zipkin.

Prerequisites

Before you begin, you’ll need to have the following installed and configured on your PATH:

I will also assume that you are logged into the Bluemix CLI and the Bluemix Container Registry CLI.  You should also have a Kubernetes cluster created in Bluemix and configured with the Kubernetes CLI.

Building the Zipkin Application

This application will act as our Zipkin server.  Both of the other apps we build will send their span data here, where it will be visualized on the Zipkin dashboard.

Setup Project

To begin, let’s create a project with the following structure:

spring-sleuth/zipkin-app/src/main/java/application/

Next create a pom.xml file with the following contents:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>com.application</groupId>
    <artifactId>zipkin-app</artifactId>
    <version>1.0-SNAPSHOT</version>

    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>1.5.2.RELEASE</version>
    </parent>

    <dependencyManagement>
        <dependencies>
            <dependency>
                <groupId>org.springframework.cloud</groupId>
                <artifactId>spring-cloud-dependencies</artifactId>
                <version>Dalston.SR1</version>
                <type>pom</type>
                <scope>import</scope>
            </dependency>
        </dependencies>
    </dependencyManagement>

    <dependencies>
        <dependency>
            <groupId>io.zipkin.java</groupId>
            <artifactId>zipkin-server</artifactId>
        </dependency>
        <dependency>
            <groupId>io.zipkin.java</groupId>
            <artifactId>zipkin-autoconfigure-ui</artifactId>
            <scope>runtime</scope>
        </dependency>
    </dependencies>

    <properties>
        <java.version>1.8</java.version>
    </properties>

    <build>
        <plugins>
            <plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
            </plugin>
        </plugins>
    </build>
</project>

Create Zipkin Server

This application will be very simple, just a single class for us to build!  Create an Application.java file with the following contents:

package application;

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import zipkin.server.EnableZipkinServer;

@SpringBootApplication
@EnableZipkinServer
public class Application {

    public static void main(String[] args) {
        SpringApplication.run(Application.class, args);
    }
}

Create Docker Image

If you’ve followed me to this point, you should have a complete Zipkin server application ready to deploy to Bluemix Kubernetes.  But first, we need to build a Docker image and push it to the Bluemix container registry.

Create a Dockerfile with the following contents:

FROM java:8
COPY /target/zipkin-app-1.0-SNAPSHOT.jar /zipkin-app-1.0-SNAPSHOT.jar
EXPOSE 8080
ENTRYPOINT java -jar /zipkin-app-1.0-SNAPSHOT.jar

Then execute the following commands in the same directory as your Dockerfile:

$ mvn package
...
$ docker build . -t registry.ng.bluemix.net/<namespace>/zipkin-app:latest
...
$ docker push registry.ng.bluemix.net/<namespace>/zipkin-app:latest
...

Note: Be sure to replace with your own private Bluemix namespace.  For help on creating a Bluemix namespace, see the Bluemix instructions.

You now have a Spring Zipkin server ready to be deployed to Kubernetes!  But before we do that, we need to build a couple applications that will produce spans.

Building the Delay Application

In this section, we’ll create a Spring REST application that takes a given amount of time to respond.  You probably wouldn’t want to do this in practice, but it will be useful to demonstrate what happens when a traced service takes certain durations to respond.

Setup Project

To begin, let’s create a project with the following structure:

spring-sleuth/delay-app/src/main/java/application/

Next create a pom.xml file with the following contents:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>com.application</groupId>
    <artifactId>delay-app</artifactId>
    <version>1.0-SNAPSHOT</version>

    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>1.5.2.RELEASE</version>
    </parent>

    <dependencyManagement>
        <dependencies>
            <dependency>
                <groupId>org.springframework.cloud</groupId>
                <artifactId>spring-cloud-dependencies</artifactId>
                <version>Dalston.SR1</version>
                <type>pom</type>
                <scope>import</scope>
            </dependency>
        </dependencies>
    </dependencyManagement>

    <dependencies>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-zipkin</artifactId>
        </dependency>
    </dependencies>

    <properties>
        <java.version>1.8</java.version>
    </properties>


    <build>
        <plugins>
            <plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
            </plugin>
        </plugins>
    </build>
</project>

Create Delay Application

Let’s start with the main class, where most of the action happens for this app.  Create an Application.java file with the following contents:

package application;

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.cloud.sleuth.annotation.NewSpan;
import org.springframework.http.HttpStatus;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestMethod;
import org.springframework.web.bind.annotation.RestController;

@SpringBootApplication
@RestController
public class Application {

    public static void main(String[] args) {
        SpringApplication.run(Application.class, args);
    }

    @NewSpan
    @RequestMapping(value = "/sleep/{time}", method = RequestMethod.GET)
    public ResponseEntity sleep(@PathVariable int time){
        try {
            Thread.sleep(time);

            return new ResponseEntity(HttpStatus.OK);
        } catch (InterruptedException e) {
            e.printStackTrace();
        }

        return new ResponseEntity(HttpStatus.INTERNAL_SERVER_ERROR);
    }
}

In this class, we setup a Spring REST app with a single endpoint: sleep.  That endpoint takes a time duration (in milliseconds) and waits before responding with OK.

On the sleep method, there is an annotation you may not be familiar with: @NewSpan.  This is a Sleuth annotation that tells our app to collect data on this method, such as when it is called, what parameters were passed, time of completion, etc.  We’ll see this information in the Zipkin dashboard near the end of this post.

Now the application is collecting span data, but where will it send it?  We need to connect our delay app to the Zipkin app created previously.  Create an application.properties file with the following contents:

spring.zipkin.baseUrl=http://zipkin-service:8080
spring.sleuth.sampler.percentage=1.0

The Zipkin URL is set to http://zipkin-service:8080.  This will be a service we create when we deploy our apps to Bluemix Kubernetes, near the end of this post.

Next, we’ll set a name for this application.  Create a bootstrap.properties file with the following contents:

spring.application.name=delay-app

Naming this app, will make it easier to distinguish between the spans created by multiple applications on the Zipkin dashboard.

Create Docker Image

If you’ve followed me to this point, you should have completed the delay Spring application. Now we will build another Docker image and push it to the Bluemix container registry.

Create a Dockerfile with the following contents:

FROM java:8
COPY /target/delay-app-1.0-SNAPSHOT.jar /delay-app-1.0-SNAPSHOT.jar
EXPOSE 8080
ENTRYPOINT java -jar /delay-app-1.0-SNAPSHOT.jar

Then execute the following commands in the same directory as your Dockerfile:

$ mvn package
...
$ docker build . -t registry.ng.bluemix.net/<namespace>/delay-app:latest
...
$ docker push registry.ng.bluemix.net/<namespace>/delay-app:latest
...

Building the Feign Application

In this section, we’ll create a Spring REST application that uses a FeignClient with Hystrix protection to make requests to our delay app.

Setup Project

To begin, let’s create a project with the following structure:

spring-sleuth/feign-app/src/main/java/application/

Next create a pom.xml file with the following contents:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>com.application</groupId>
    <artifactId>feign-app</artifactId>
    <version>1.0-SNAPSHOT</version>

    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>1.5.2.RELEASE</version>
    </parent>

    <dependencyManagement>
        <dependencies>
            <dependency>
                <groupId>org.springframework.cloud</groupId>
                <artifactId>spring-cloud-dependencies</artifactId>
                <version>Dalston.SR1</version>
                <type>pom</type>
                <scope>import</scope>
            </dependency>
        </dependencies>
    </dependencyManagement>

    <dependencies>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-config</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-feign</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-hystrix</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-zipkin</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
        </dependency>
    </dependencies>

    <properties>
        <java.version>1.8</java.version>
    </properties>

    <build>
        <plugins>
            <plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
            </plugin>
        </plugins>
    </build>
</project>

Create Feign Application

Let’s start with the FeignClient class.  Create a DelayClient.java file with the following contents:

package application;

import org.springframework.cloud.netflix.feign.FeignClient;
import org.springframework.cloud.sleuth.annotation.NewSpan;
import org.springframework.http.HttpStatus;
import org.springframework.http.ResponseEntity;
import org.springframework.stereotype.Component;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestMethod;

@FeignClient(name = "delay", url = "http://delay-service:8080", fallback = DelayClient.DelayClientFallback.class)
public interface DelayClient {

    @RequestMapping(value = "/sleep/{time}", method = RequestMethod.GET)
    ResponseEntity sleep(@PathVariable("time") int time);

    @Component
    class DelayClientFallback implements DelayClient {

        @NewSpan
        @Override
        public ResponseEntity sleep(int time) {
            return new ResponseEntity("Fallback\n", HttpStatus.OK);
        }
    }
}

In this class, we setup a FeignClient and set it to the URL where our delay app will be available. We also setup a Hystrix fallback for the sleep method, which will allow us to explore how Sleuth handles certain faulty requests. If you’re unfamiliar with the contents of this class, you can read my post on Connecting a Spring Cloud application to Cloudant Service with Feign and Hystrix.

Next let’s setup the main class, which will contain a REST endpoint for calling the FeignClient method. Create an Application.java class with the following contents:

package application;

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.cloud.client.circuitbreaker.EnableCircuitBreaker;
import org.springframework.cloud.netflix.feign.EnableFeignClients;
import org.springframework.cloud.sleuth.annotation.NewSpan;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.RequestMapping;

@SpringBootApplication
@EnableFeignClients
@EnableCircuitBreaker
@RestController
public class Application {

    @Autowired
    private DelayClient delayClient;

    public static void main(String[] args) {
        SpringApplication.run(Application.class, args);
    }

    @NewSpan
    @RequestMapping("/feign/{time}")
    public ResponseEntity feign(@PathVariable int time) {
        return delayClient.sleep(time);
    }
}

Now we need to configure our app for Zipkin and Hystrix.  Create an application.properties file with the following contents:

spring.zipkin.baseUrl=http://zipkin-service:8080
spring.sleuth.sampler.percentage=1.0

feign.hystrix.enabled=true
hystrix.command.default.execution.isolation.thread.timeoutInMilliseconds=2000
hystrix.command.default.circuitBreaker.enabled=true
hystrix.command.default.circuitBreaker.requestVolumeThreshold=2

Next, we’ll set a name for this application.  Create an bootstrap.properties file with the following contents:

spring.application.name=feign-app

Create Docker Image

If you’ve followed me to this point, you should have completed the feign Spring application. Now we will build another Docker image and push it to the Bluemix container registry.

Create a Dockerfile with the following contents:

FROM java:8
COPY /target/feign-app-1.0-SNAPSHOT.jar /feign-app-1.0-SNAPSHOT.jar
EXPOSE 8080
ENTRYPOINT java -jar /feign-app-1.0-SNAPSHOT.jar

Then execute the following commands in the same directory as your Dockerfile:

$ mvn package
...
$ docker build . -t registry.ng.bluemix.net/<namespace>/feign-app:latest
...
$ docker push registry.ng.bluemix.net/<namespace>/feign-app:latest
...

Deploying to Bluemix Kubernetes

Create deployment files

Now let’s create a couple YAML files to deploy to our Kubernetes cluster.  We will need files to deploy our application pods and the services that access them.

First, create a pods.yaml file with the following contents:

apiVersion: v1
kind: Pod
metadata:
  name: zipkin-app
  labels:
    name: zipkin-app
spec:
  containers:
    - name: zipkin-app
      image: registry.ng.bluemix.net/<namespace>/zipkin-app:latest
      ports:
        - containerPort: 8080

---

apiVersion: v1
kind: Pod
metadata:
  name: delay-app
  labels:
    name: delay-app
spec:
  containers:
    - name: delay-app
      image: registry.ng.bluemix.net/jeff/delay-app:latest
      ports:
        - containerPort: 8080

---

apiVersion: v1
kind: Pod
metadata:
  name: feign-app
  labels:
    name: feign-app
spec:
  containers:
    - name: feign-app
      image: registry.ng.bluemix.net/jeff/feign-app:latest
      ports:
        - containerPort: 8080

In this file we are creating a pod for each of the applications we built and exposing the default Spring port on each. Next, create a services.yaml file with the following contents:

apiVersion: v1
kind: Service
metadata:
  name: zipkin-service
  namespace: default
spec:
  ports:
    - port: 8080
  selector:
    name: zipkin-app

---

apiVersion: v1
kind: Service
metadata:
  name: delay-service
  namespace: default
spec:
  ports:
    - port: 8080
  selector:
    name: delay-app

---

apiVersion: v1
kind: Service
metadata:
  name: feign-service
  namespace: default
spec:
  type: NodePort
  ports:
    - port: 8080
  selector:
    name: feign-app

Deploy to cluster

Now we are ready to run our Spring applications on our Bluemix Kubernetes cluster!  Create a proxy connection to your cluster with the following command:

$ kubectl proxy

Then navigate to the Kubernetes dashboard at: 127.0.0.1:8001/ui.

From the dashboard, click CREATE in the top-right corner, then select the Upload a YAML or JSON file option, and choose your pods.yaml file.  Upload this file with the UPLOAD button.  Then follow the same procedure with your services.yaml file.

After your pods have finished starting up, you are ready to continue.

You will need to find where the Zipkin and feign services are being exposed. To find the public node IP, click Nodes on the left sidebar and use the name of the node that your pod is running on (there should only be one node if you are using a Lite cluster plan). To find the service NodePorts, click Services on the left sidebar and find the port number (in the range 30000-32767) associated with the “zipkin-service” and “feign-service” services.

In a terminal, use the following command to make a request to the feign service. The URL will be formed by the public node IP and feign service NodePort.

$ curl http://168.1.140.71:31069/feign/1000

This command should take about a second to complete, with no response. Next we will look at the spans generated on the Zipkin dashboard.

In your web browser, visit the address formed by your public node IP and Zipkin service NodePort. For example, I visited the following URL and saw this page:

Sort the entries by “Longest First,” then select the largest one. You should see something similar to the following:

These are the spans created by our application methods. If you click on a particular span, you will see some other information that was gathered by Sleuth.

The last thing we will test is what happens when a Hystrix fallback is triggered due to a request timeout. Use the following command to cause the feign app to timeout:

$ curl http://168.1.140.71:31069/feign/3000

Now if you return to the the main Zipkin dashboard page, there should be a new longest entry. If you explore the spans in this entry, you will see exactly where Hystrix resorted to its fallback, as well as the completion of the method that took too long.

Conclusion

If you followed along, you now have a basic set of applications demonstrating Spring Sleuth, with spans visualized through the Zipkin dashboard.  Although we used a “fake” service in this post, you should now be able to plug in whatever services you need and incorporate tracing in a similar way.

Add Comment
No Comments

Leave a Reply

Your email address will not be published.Required fields are marked *

More How-tos Stories

Vaadin Bakery: Jump-start your business web app

Vaadin Bakery App Starter is a proven full-stack reference application you can use as a starting point for many serious business web apps. It contains many commonly needed features, like RDBMS database accessed using solid JPA+EJB (or Spring) -based persistency and business layer, mindful authentication and authorization, and a UI code structure suitable for non-trivial, large-scale business applications.

Continue reading

Financial transaction compliance made easy with Yantra

Yantra transformed into an award winning Fintech powerhouse. Yantra Financial Technologies was recognized as a “Company to Watch” as part of the 2016 FinTech Forward rankings released by American Banker and BAI.

Continue reading

Building financial insight starts with WealthEngine

The WealthEngine API lets you look up the net worth and financial capacity of almost anyone in the United States, in real time, giving you insight into the wallet share of clients and prospects. Simple to use, it's a restful API that returns JSON.

Continue reading