Everybody loves REST APIs. With such interfaces, developers can easily hook into external systems and exchange data or trigger events. In this article, we’ll look into a mechanism that will allow anyone to both specify and rely on a strict API specification.


Of course, you can read this article without ever leaving the browser, but it’s a lot more fun when following along. Here’s the list of things you might want to have:

  • wget or a similar tool to download things from the internet
  • A current Java runtime environment—the java command should be on your path
  • A Golang version of 1.12 on your path
  • A text editor of your choice

So, what’s the problem with REST APIs?

Whenever we’re working with foreign REST APIs, we need to rely on the public documentation that the vendor supplies. These documents need to be as accurate as possible. As time goes by, code will inevitably diverge from your documentation as you’ll most probably enhance the feature set and fix conceptual issues.

With the IBM Instana™ platform, we have chosen to evolve our REST API over time, and without a big bang. To be able to expose accurate documentation, we automatically publish machine-generated documentation on every release to our GitHub pages documentation.

In the document, there’s nothing fancy—an HTML documentation, with some code snippets and endpoint specifications—and time to create code resembling our model into your consumer code, right? Not quite. There’s a quick and easy path to consume our API from within your system without wrestling too much with our data formats.

Enter OpenAPI

OpenAPI—the artist formerly known as “Swagger Specification”—is a project that aims to provide a path to happiness for both API consumers and vendors. By designing the specification rather than the endpoints directly, it aims at having a stable and binding contract between vendors and consumers.

What does such a specification look like? Here we have a shortened version of a specification for the infamous “PetShop” application:

openapi: "3.0.0"
  version: 1.0.0
  title: Swagger Petstore
    name: MIT
  - url: http://petstore.swagger.io/v1
      summary: List all pets
      operationId: listPets
        - pets
        - name: limit
          in: query
          description: How many items to return at one time (max 100)
          required: false
            type: integer
            format: int32
          description: A paged array of pets
              description: A link to the next page of responses
                type: string
                $ref: "#/components/schemas/Pets"
      type: object
        - id
        - name
          type: integer
          format: int64
          type: string
          type: string
      type: array
        $ref: "#/components/schemas/Pet"

You might already have glimpsed at the possible consequences for the API in question. It says we have an endpoint /pets with a possible GET HTTP operation on it that returns a paginated list of Pets. When we now look up the Pets object in the components section, we can quickly infer that it’s an array of Pet objects and the Pet object is specified, as well, and we can see that a Pet has mandatory name and id properties with associate types.

If we’re able to read such specifications with our own eyes, why can’t machines?

Enter OpenAPI Generator

In the OpenAPI ecosystem, there are multiple projects focused on processing specification documents. These projects range from validating documents to creating mock servers and testing utilities to generating client and even server code. The one we’ll focus on in this article is the OpenAPI Generator project.

The OpenAPI Generator project focuses on providing a convenient way to generate code you can work with from a given specification document from a single command.

To spice things up a bit, we’ll be working with our API specification. You can retrieve it through the generated documentation, YAML version or JSON version. This document is updated regularly within our release cycle, and you should be able to work with it right out of the box.

Enter Golang

The Go language is great for creating small binaries for almost every occasion in a straightforward fashion. We’ll use that language and we require at least version `1.12` for proper modules support.

First, we want to create our project directory. Download the OpenAPI Generator and see what it can do for us:

# create and enter the project directory
$ mkdir instana-golang-example && cd $_
# download the jar
$ wget https://repo1.maven.org/maven2/org/openapitools/openapi-generator-cli/5.0.0-beta/openapi-generator-cli-5.0.0-beta.jar \
  -O openapi-generator-cli.jar

# take peek at the different generators it offers us
$ java -jar openapi-generator-cli.jar

An impressive list, right? Next, we’ll use that jar file to generate a Go client for the IBM Instana REST API:

# download the Instana openapi spec
$ mkdir resources && wget https://instana.github.io/openapi/openapi.yaml -O resources/openapi.yaml
# generated code is ugly, we'll use "gofmt" to format it in the same step
$ GO_POST_PROCESS_FILE="gofmt -s -w" java -jar openapi-generator-cli.jar generate -i resources/openapi.yaml -g go \
    -o pkg/instana \
    --additional-properties packageName=instana \
    --additional-properties isGoSubmodule=true \
    --additional-properties packageVersion=1.185.824 \
    --type-mappings=object=interface{} \
    --enable-post-process-file \

Done. Don’t believe me? Let’s have a look into the generated documentation: cat pkg/instana/README.md

We can now use it in our tiny new tool. Let’s create a go.mod file in the current directory to include the library in our project:

module github.com/instana/instana-openapi-golang-example

require github.com/instana/instana-openapi-golang-example/pkg/instana v1.185.824

replace github.com/instana/instana-openapi-golang-example/pkg/instana v1.185.824 => ./pkg/instana

Cool. We just linked our local API client into the Go modules dependency management. Go modules are awesome.

To show you how to use the client, we can create a small program that will query all the available search fields you can use in the IBM Instana platform.

Let’s create a main.go file in our current directory:

package main

import (


func main() {
	// grab settings from environment
	apiKey := os.Getenv("INSTANA_KEY")
	tenant := os.Getenv("INSTANA_TENANT")
	unit := os.Getenv("INSTANA_UNIT")

	// create golang client specific configuration
	configuration := instana.NewConfiguration()
	host := fmt.Sprintf("%s-%s.instana.io", unit, tenant)
	configuration.Host = host
	configuration.BasePath = fmt.Sprintf("https://%s", host)

	client := instana.NewAPIClient(configuration)

	// Instana uses the `apiToken` prefix in the `Authorization` header
	auth := context.WithValue(context.Background(), instana.ContextAPIKey, instana.APIKey{
		Key:    apiKey,
		Prefix: "apiToken",

	searchFieldResults, _, err := client.InfrastructureCatalogApi.GetInfrastructureCatalogSearchFields(auth)
	if err != nil {


	fmt.Println("Available search fields supported by Instana:")
	for _, field := range searchFieldResults {
		fmt.Println(fmt.Sprintf("%s (%s) - %s", field.Keyword, field.Context, field.Description))

Next, we need an API token. To get yourself a shiny new token, head to your IBM Instana instance, and go to Settings > API Tokens > Add API Token.

Side note: We need the tenant and unit from your environment. When using our SaaS offering, you can get them from your environment’s URL. If you’re accessing your environment through https://qa-acme.instana.io, then acme is the tenant and qa is the unit.

That’s it. Let’s run our code and see what search fields we can use with the IBM Instana platform:

$ INSTANA_TENANT=acme INSTANA_UNIT=qa INSTANA_KEY=$your_token$ go run main.go

Available search fields supported by Instana:
entity.kubernetes.pod.name (entities) - Kubernetes Pod Name
event.specification.id (events) - Event specification ID
entity.ping.target (entities) - Ping target
entity.azure.service.sqlserver.name (entities) - The name of the SQL database server
entity.lxc.ip (entities) - LXC container IP
entity.jboss.serverName (entities) - JBoss Server Name

Making use of it

Listing the available tags is one thing, but nothing fancy, right? We want to live up to the hype and get metrics from a specific Kubernetes pod. Specifically, the CPU requests over time.

To facilitate this process, we need to find what we’re searching for. The IBM Instana platform organizes metrics by plug-ins. To see what plug-ins are available, we can query them:

// search for the kubernetes pod plugin
// https://instana.github.io/openapi/#operation/getInfrastructureCatalogPlugins
plugins, _, err := client.InfrastructureCatalogApi.GetInfrastructureCatalogPlugins(auth)
if err != nil {
	println(fmt.Errorf("Error reading plugins: %s", err))


println("Let's find ourselves some kubernetes pods!")

// search for kubernetes plugins:
for _, plugin := range plugins {
	if strings.Contains(plugin.Plugin, "kubernetes") {
		fmt.Printf("Found Kubernetes plugin %s with ID %s\n", plugin.Label, plugin.Plugin)

The output will be very similar to this:

Metric for plugin `kubernetesPod`: Containers (ID: container_count)
Metric for plugin `kubernetesPod`: Memory Requests (ID: memoryRequests)
Metric for plugin `kubernetesPod`: CPU Limits (ID: cpuLimits)
Metric for plugin `kubernetesPod`: CPU Requests (ID: cpuRequests)
Metric for plugin `kubernetesPod`: Restarts (ID: restartCount)
Metric for plugin `kubernetesPod`: Memory Limits (ID: memoryLimits)

There you go. Our metric ID for getting CPU requests from the “kubernetesPod” plugin is “cpuRequests”.

Finding your snapshot

Next thing, knowing what metric from which plug-in we want, we need to find Kubernetes pods in our infrastructure inventory. The IBM Instana platform organizes infrastructure changes over time in a concept called “snapshot.” We need to find a snapshot that contains “kubernetesPod” entities and then we can go query those metrics.

When you deploy our IBM Instana agent through one of the supported methods to your Kubernetes cluster, the agent pods themselves will have names in the form of “instana-agent-ID”. Knowing this information, we can search for all snapshots that contain entities with values of instana-agent-* in the entity.kubernetes.pod.name field.

Do the same as you would query through the Dynamic Focus query:

// let's find all the Snapshots that involve Kubernetes pods!
// https://instana.github.io/openapi/#operation/getSnapshots
snapshotsWithKubernetesPods, _, err := client.InfrastructureMetricsApi.GetSnapshots(auth, &instana.GetSnapshotsOpts{
	Plugin: optional.NewString(kubernetesPodPluginID),

	// We know that clusters, when monitored with Instana usually have pods with a name of `instana-agent-*`
	Query: optional.NewString("entity.kubernetes.pod.name:instana-agent*"),

	// We can travel through time and query data from entities that are no more!
	Offline: optional.NewBool(true),

	// Instana uses Milliseconds accross the board
	WindowSize: optional.NewInt64(7 * 86400 * 1000),
	To:         optional.NewInt64(time.Now().Unix() * int64(time.Millisecond)),
if err != nil {
	println(fmt.Errorf("Error reading snapshots: %s", err))


for _, snapshot := range snapshotsWithKubernetesPods.Items {
	fmt.Printf("Kubernetes Pod %s on Host %s with snapshot ID %s \n", snapshot.Label, snapshot.Host, snapshot.SnapshotId)

The output will look like this:

Let's find a Kubernetes Pod that contains an Instana Agent
Kubernetes Pod instana-agent/instana-agent-1-fn69h (pod) on Host with snapshot ID fkHp5kkCvpBonSAkyZD03GagAL4
Kubernetes Pod instana-agent/instana-agent-1-7jlwd (pod) on Host with snapshot ID 2sCSGsxJGUJB3mpaQ7SVOHJlMmY
Kubernetes Pod instana-agent/instana-agent-1-7rqvm (pod) on Host with snapshot ID FU7AaxMkghV9vsoB05AEXdrZqpE

Retrieving the metrics

Now that we know the snapshot IDs, we can go and fetch the metrics from those snapshots:

println("Let's put everything together: Querying the cpuRequests pod metrics from a specific snapshot")

metricID := "cpuRequests"
metricsQuery := instana.GetCombinedMetrics{
	Plugin: kubernetesPodPluginID,
	Metrics: []string{

	SnapshotIds: []string{

	TimeFrame: instana.TimeFrame{
		To:         time.Now().Unix() * 1000,
		WindowSize: 300000,

	// 5 Minutes
	Rollup: 60,

metricsResult, _, err := client.InfrastructureMetricsApi.GetInfrastructureMetrics(auth, &instana.GetInfrastructureMetricsOpts{
	GetCombinedMetrics: optional.NewInterface(metricsQuery),

if err != nil {
	println(fmt.Errorf("Error reading metrics: %s", err.(instana.GenericOpenAPIError)))


for _, metric := range metricsResult.Items {
	for _, bracket := range metric.Metrics[metricID] {
		parsedTime := time.Unix(0, int64(bracket[0])*int64(time.Millisecond))
		fmt.Printf("CPU requests of Kubernetes Pod %s at %s: %f\n", snapshotsWithKubernetesPods.Items[0].Label, parsedTime, bracket[1])

The output will look like this:

Let's put everything together: Querying the cpuRequests pod metrics from a specific snapshot
CPU requests of Kubernetes Pod instana-agent/instana-agent-1-fn69h (pod) at 2020-09-03 10:56:06.656 +0000 UTC: 0.599900
CPU requests of Kubernetes Pod instana-agent/instana-agent-1-fn69h (pod) at 2020-09-03 10:58:17.728 +0000 UTC: 0.599900
CPU requests of Kubernetes Pod instana-agent/instana-agent-1-fn69h (pod) at 2020-09-03 10:58:17.728 +0000 UTC: 0.599900
CPU requests of Kubernetes Pod instana-agent/instana-agent-1-fn69h (pod) at 2020-09-03 11:00:28.8 +0000 UTC: 0.599900
CPU requests of Kubernetes Pod instana-agent/instana-agent-1-fn69h (pod) at 2020-09-03 11:00:28.8 +0000 UTC: 0.599900
CPU requests of Kubernetes Pod instana-agent/instana-agent-1-fn69h (pod) at 2020-09-03 11:02:39.872 +0000 UTC: 0.599900

Having read up to here, you learned how you can:

  • Create or, rather, generate a API client in Golang through an OpenAPI specification.
  • Use that client to query the IBM Instana REST API in a type-safe way.
  • Resolve metric values from a specific item from the IBM Instana platform’s vast, automatically created infrastructure store.


In this article, we learned how to generate an API client for a specific programming environment based on an OpenAPI specification that’s provided by a third party. This method is adaptable for many different projects and languages. An upside of this approach is that the contract is basically also the code that’s being executed to communicate with the remote API.

In case of the IBM Instana REST API, the OpenAPI makes it easy for us, and our clients to version our API specification and separate our on-premises product and SaaS offering, so you can always generate a client library for your specific environment.

The complete code is available on GitHub.

Get started with IBM Instana


More from IBM Instana

Observing Camunda environments with IBM Instana Business Monitoring

3 min read - Organizations today struggle to detect, identify and act on business operations incidents. The gap between business and IT continues to grow, leaving orgs unable to link IT outages to business impact.  Site reliability engineers (SREs) want to understand business impact to better prioritize their work but don’t have a way of monitoring business KPIs. They struggle to link IT outages to business impacts because data is often siloed and knowledge is tribal. It forces teams into a highly reactive mode…

Buying APM was a good decision (so is getting rid of it)

4 min read - For a long time, there wasn’t a good standard definition of observability that encompassed organizational needs while keeping the spirit of IT monitoring intact. Eventually, the concept of “Observability = Metrics + Traces + Logs” became the de facto definition. That’s nice, but to understand what observability should be, you must consider the characteristics of modern applications: Changes in how they’re developed, deployed and operated The blurring of lines between application code and infrastructure New architectures and technologies like Docker,…

Debunking observability myths – Part 5: You can create an observable system without observability-driven automation

3 min read - In our blog series, we’ve debunked the following observability myths so far: Part 1: You can skip monitoring and rely solely on logs Part 2: Observability is built exclusively for SREs Part 3: Observability is only relevant and beneficial for large-scale systems or complex architectures Part 4: Observability is always expensive In this post, we'll tackle another fallacy that limits the potential of observability—that you can create an observable system without observability driven by automation. Why is this a myth? The notion that…

Top 8 APM metrics that IT teams use to monitor their apps

5 min read - A superior customer experience (CX) is built on accurate and timely application performance monitoring (APM) metrics. You can’t fine-tune your apps or system to improve CX until you know what the problem is or where the opportunities are. APM solutions typically provide a centralized dashboard to aggregate real-time performance metrics and insights to be analyzed and compared. They also establish baselines to alert system administrators to deviations that indicate actual or potential performance issues. IT teams, DevOps and site reliability…