Everybody loves REST APIs. With such interfaces, developers can easily hook into external systems and exchange data or trigger events. In this article, we’ll look into a mechanism that will allow anyone to both specify and rely on a strict API specification.

Prerequisites

Of course, you can read this article without ever leaving the browser, but it’s a lot more fun when following along. Here’s the list of things you might want to have:

• wget or a similar tool to download things from the internet
• A current Java runtime environment—the java command should be on your path
• A Golang version of 1.12 on your path
• A text editor of your choice

So, what’s the problem with REST APIs?

Whenever we’re working with foreign REST APIs, we need to rely on the public documentation that the vendor supplies. These documents need to be as accurate as possible. As time goes by, code will inevitably diverge from your documentation as you’ll most probably enhance the feature set and fix conceptual issues.

With the IBM Instana™ platform, we have chosen to evolve our REST API over time, and without a big bang. To be able to expose accurate documentation, we automatically publish machine-generated documentation on every release to our GitHub pages documentation.

In the document, there’s nothing fancy—an HTML documentation, with some code snippets and endpoint specifications—and time to create code resembling our model into your consumer code, right? Not quite. There’s a quick and easy path to consume our API from within your system without wrestling too much with our data formats.

Enter OpenAPI

OpenAPI—the artist formerly known as “Swagger Specification”—is a project that aims to provide a path to happiness for both API consumers and vendors. By designing the specification rather than the endpoints directly, it aims at having a stable and binding contract between vendors and consumers.

What does such a specification look like? Here we have a shortened version of a specification for the infamous “PetShop” application:

openapi: "3.0.0"
info:
  version: 1.0.0
  title: Swagger Petstore
  license:
    name: MIT
servers:
  - url: http://petstore.swagger.io/v1
paths:
  /pets:
    get:
      summary: List all pets
      operationId: listPets
      tags:
        - pets
      parameters:
        - name: limit
          in: query
          description: How many items to return at one time (max 100)
          required: false
          schema:
            type: integer
            format: int32
      responses:
        '200':
          description: A paged array of pets
          headers:
            x-next:
              description: A link to the next page of responses
              schema:
                type: string
          content:
            application/json:    
              schema:
                $ref: "#/components/schemas/Pets"
components:
  schemas:
    Pet:
      type: object
      required:
        - id
        - name
      properties:
        id:
          type: integer
          format: int64
        name:
          type: string
        tag:
          type: string
    Pets:
      type: array
      items:
        $ref: "#/components/schemas/Pet"

You might already have glimpsed at the possible consequences for the API in question. It says we have an endpoint /pets with a possible GET HTTP operation on it that returns a paginated list of Pets. When we now look up the Pets object in the components section, we can quickly infer that it’s an array of Pet objects and the Pet object is specified, as well, and we can see that a Pet has mandatory name and id properties with associate types.

If we’re able to read such specifications with our own eyes, why can’t machines?

Enter OpenAPI Generator

In the OpenAPI ecosystem, there are multiple projects focused on processing specification documents. These projects range from validating documents to creating mock servers and testing utilities to generating client and even server code. The one we’ll focus on in this article is the OpenAPI Generator project.

The OpenAPI Generator project focuses on providing a convenient way to generate code you can work with from a given specification document from a single command.

To spice things up a bit, we’ll be working with our API specification. You can retrieve it through the generated documentation, YAML version or JSON version. This document is updated regularly within our release cycle, and you should be able to work with it right out of the box.

Enter Golang

The Go language is great for creating small binaries for almost every occasion in a straightforward fashion. We’ll use that language and we require at least version `1.12` for proper modules support.

First, we want to create our project directory. Download the OpenAPI Generator and see what it can do for us:

# create and enter the project directory
$ mkdir instana-golang-example && cd $_
# download the jar
$ wget https://repo1.maven.org/maven2/org/openapitools/openapi-generator-cli/5.0.0-beta/openapi-generator-cli-5.0.0-beta.jar \
  -O openapi-generator-cli.jar

# take peek at the different generators it offers us
$ java -jar openapi-generator-cli.jar

An impressive list, right? Next, we’ll use that jar file to generate a Go client for the IBM Instana REST API:

# download the Instana openapi spec
$ mkdir resources && wget https://instana.github.io/openapi/openapi.yaml -O resources/openapi.yaml
# generated code is ugly, we'll use "gofmt" to format it in the same step
$ GO_POST_PROCESS_FILE="gofmt -s -w" java -jar openapi-generator-cli.jar generate -i resources/openapi.yaml -g go \
    -o pkg/instana \
    --additional-properties packageName=instana \
    --additional-properties isGoSubmodule=true \
    --additional-properties packageVersion=1.185.824 \
    --type-mappings=object=interface{} \
    --enable-post-process-file \
    --skip-validate-spec

Done. Don’t believe me? Let’s have a look into the generated documentation: cat pkg/instana/README.md

We can now use it in our tiny new tool. Let’s create a go.mod file in the current directory to include the library in our project:

module github.com/instana/instana-openapi-golang-example

require github.com/instana/instana-openapi-golang-example/pkg/instana v1.185.824

replace github.com/instana/instana-openapi-golang-example/pkg/instana v1.185.824 => ./pkg/instana

Cool. We just linked our local API client into the Go modules dependency management. Go modules are awesome.

To show you how to use the client, we can create a small program that will query all the available search fields you can use in the IBM Instana platform.

Let’s create a main.go file in our current directory:

package main

import (
	"context"
	"fmt"
	"os"
	"strings"
	"time"

	"github.com/antihax/optional"
	"github.com/instana/instana-openapi-golang-example/pkg/instana"
)

func main() {
	// grab settings from environment
	apiKey := os.Getenv("INSTANA_KEY")
	tenant := os.Getenv("INSTANA_TENANT")
	unit := os.Getenv("INSTANA_UNIT")

	// create golang client specific configuration
	configuration := instana.NewConfiguration()
	host := fmt.Sprintf("%s-%s.instana.io", unit, tenant)
	configuration.Host = host
	configuration.BasePath = fmt.Sprintf("https://%s", host)

	client := instana.NewAPIClient(configuration)

	// Instana uses the `apiToken` prefix in the `Authorization` header
	auth := context.WithValue(context.Background(), instana.ContextAPIKey, instana.APIKey{
		Key:    apiKey,
		Prefix: "apiToken",
	})

	searchFieldResults, _, err := client.InfrastructureCatalogApi.GetInfrastructureCatalogSearchFields(auth)
	if err != nil {
		fmt.Println(err)

		os.Exit(1)
	}

	fmt.Println("Available search fields supported by Instana:")
	for _, field := range searchFieldResults {
		fmt.Println(fmt.Sprintf("%s (%s) - %s", field.Keyword, field.Context, field.Description))
	}
}

Next, we need an API token. To get yourself a shiny new token, head to your IBM Instana instance, and go to Settings > API Tokens > Add API Token.

Side note: We need the tenant and unit from your environment. When using our SaaS offering, you can get them from your environment’s URL. If you’re accessing your environment through https://qa-acme.instana.io, then acme is the tenant and qa is the unit.

That’s it. Let’s run our code and see what search fields we can use with the IBM Instana platform:

$ INSTANA_TENANT=acme INSTANA_UNIT=qa INSTANA_KEY=$your_token$ go run main.go

Available search fields supported by Instana:
entity.kubernetes.pod.name (entities) - Kubernetes Pod Name
event.specification.id (events) - Event specification ID
entity.ping.target (entities) - Ping target
entity.azure.service.sqlserver.name (entities) - The name of the SQL database server
entity.lxc.ip (entities) - LXC container IP
entity.jboss.serverName (entities) - JBoss Server Name
...

Making use of it

Listing the available tags is one thing, but nothing fancy, right? We want to live up to the hype and get metrics from a specific Kubernetes pod. Specifically, the CPU requests over time.

To facilitate this process, we need to find what we’re searching for. The IBM Instana platform organizes metrics by plug-ins. To see what plug-ins are available, we can query them:

// search for the kubernetes pod plugin
// https://instana.github.io/openapi/#operation/getInfrastructureCatalogPlugins
plugins, _, err := client.InfrastructureCatalogApi.GetInfrastructureCatalogPlugins(auth)
if err != nil {
	println(fmt.Errorf("Error reading plugins: %s", err))

	os.Exit(1)
}

println("Let's find ourselves some kubernetes pods!")

// search for kubernetes plugins:
for _, plugin := range plugins {
	if strings.Contains(plugin.Plugin, "kubernetes") {
		fmt.Printf("Found Kubernetes plugin %s with ID %s\n", plugin.Label, plugin.Plugin)
	}
}

The output will be very similar to this:

Metric for plugin `kubernetesPod`: Containers (ID: container_count)
Metric for plugin `kubernetesPod`: Memory Requests (ID: memoryRequests)
Metric for plugin `kubernetesPod`: CPU Limits (ID: cpuLimits)
Metric for plugin `kubernetesPod`: CPU Requests (ID: cpuRequests)
Metric for plugin `kubernetesPod`: Restarts (ID: restartCount)
Metric for plugin `kubernetesPod`: Memory Limits (ID: memoryLimits)

There you go. Our metric ID for getting CPU requests from the “kubernetesPod” plugin is “cpuRequests”.

Finding your snapshot

Next thing, knowing what metric from which plug-in we want, we need to find Kubernetes pods in our infrastructure inventory. The IBM Instana platform organizes infrastructure changes over time in a concept called “snapshot.” We need to find a snapshot that contains “kubernetesPod” entities and then we can go query those metrics.

When you deploy our IBM Instana agent through one of the supported methods to your Kubernetes cluster, the agent pods themselves will have names in the form of “instana-agent-ID”. Knowing this information, we can search for all snapshots that contain entities with values of instana-agent-* in the entity.kubernetes.pod.name field.

Do the same as you would query through the Dynamic Focus query:

// let's find all the Snapshots that involve Kubernetes pods!
// https://instana.github.io/openapi/#operation/getSnapshots
snapshotsWithKubernetesPods, _, err := client.InfrastructureMetricsApi.GetSnapshots(auth, &instana.GetSnapshotsOpts{
	Plugin: optional.NewString(kubernetesPodPluginID),

	// We know that clusters, when monitored with Instana usually have pods with a name of `instana-agent-*`
	Query: optional.NewString("entity.kubernetes.pod.name:instana-agent*"),

	// We can travel through time and query data from entities that are no more!
	Offline: optional.NewBool(true),

	// Instana uses Milliseconds accross the board
	WindowSize: optional.NewInt64(7 * 86400 * 1000),
	To:         optional.NewInt64(time.Now().Unix() * int64(time.Millisecond)),
})
if err != nil {
	println(fmt.Errorf("Error reading snapshots: %s", err))

	os.Exit(1)
}

for _, snapshot := range snapshotsWithKubernetesPods.Items {
	fmt.Printf("Kubernetes Pod %s on Host %s with snapshot ID %s \n", snapshot.Label, snapshot.Host, snapshot.SnapshotId)
}

The output will look like this:

Let's find a Kubernetes Pod that contains an Instana Agent
Kubernetes Pod instana-agent/instana-agent-1-fn69h (pod) on Host with snapshot ID fkHp5kkCvpBonSAkyZD03GagAL4
Kubernetes Pod instana-agent/instana-agent-1-7jlwd (pod) on Host with snapshot ID 2sCSGsxJGUJB3mpaQ7SVOHJlMmY
Kubernetes Pod instana-agent/instana-agent-1-7rqvm (pod) on Host with snapshot ID FU7AaxMkghV9vsoB05AEXdrZqpE

Retrieving the metrics

Now that we know the snapshot IDs, we can go and fetch the metrics from those snapshots:

println("Let's put everything together: Querying the cpuRequests pod metrics from a specific snapshot")

metricID := "cpuRequests"
metricsQuery := instana.GetCombinedMetrics{
	Plugin: kubernetesPodPluginID,
	Metrics: []string{
		metricID,
	},

	SnapshotIds: []string{
		snapshotsWithKubernetesPods.Items[0].SnapshotId,
	},

	TimeFrame: instana.TimeFrame{
		To:         time.Now().Unix() * 1000,
		WindowSize: 300000,
	},

	// 5 Minutes
	Rollup: 60,
}

metricsResult, _, err := client.InfrastructureMetricsApi.GetInfrastructureMetrics(auth, &instana.GetInfrastructureMetricsOpts{
	GetCombinedMetrics: optional.NewInterface(metricsQuery),
})

if err != nil {
	println(fmt.Errorf("Error reading metrics: %s", err.(instana.GenericOpenAPIError)))

	os.Exit(1)
}

for _, metric := range metricsResult.Items {
	for _, bracket := range metric.Metrics[metricID] {
		parsedTime := time.Unix(0, int64(bracket[0])*int64(time.Millisecond))
		fmt.Printf("CPU requests of Kubernetes Pod %s at %s: %f\n", snapshotsWithKubernetesPods.Items[0].Label, parsedTime, bracket[1])
	}
}

The output will look like this:

Let's put everything together: Querying the cpuRequests pod metrics from a specific snapshot
CPU requests of Kubernetes Pod instana-agent/instana-agent-1-fn69h (pod) at 2020-09-03 10:56:06.656 +0000 UTC: 0.599900
CPU requests of Kubernetes Pod instana-agent/instana-agent-1-fn69h (pod) at 2020-09-03 10:58:17.728 +0000 UTC: 0.599900
CPU requests of Kubernetes Pod instana-agent/instana-agent-1-fn69h (pod) at 2020-09-03 10:58:17.728 +0000 UTC: 0.599900
CPU requests of Kubernetes Pod instana-agent/instana-agent-1-fn69h (pod) at 2020-09-03 11:00:28.8 +0000 UTC: 0.599900
CPU requests of Kubernetes Pod instana-agent/instana-agent-1-fn69h (pod) at 2020-09-03 11:00:28.8 +0000 UTC: 0.599900
CPU requests of Kubernetes Pod instana-agent/instana-agent-1-fn69h (pod) at 2020-09-03 11:02:39.872 +0000 UTC: 0.599900

Having read up to here, you learned how you can:

• Create or, rather, generate a API client in Golang through an OpenAPI specification.
• Use that client to query the IBM Instana REST API in a type-safe way.
• Resolve metric values from a specific item from the IBM Instana platform’s vast, automatically created infrastructure store.

Summary

In this article, we learned how to generate an API client for a specific programming environment based on an OpenAPI specification that’s provided by a third party. This method is adaptable for many different projects and languages. An upside of this approach is that the contract is basically also the code that’s being executed to communicate with the remote API.

In case of the IBM Instana REST API, the OpenAPI makes it easy for us, and our clients to version our API specification and separate our on-premises product and SaaS offering, so you can always generate a client library for your specific environment.

The complete code is available on GitHub.