AWS Lambda Native Tracing for Go

The Go specifics for AWS Lambda tracing is described here.

Supported Runtimes

  • go1.x using Go 1.8 or later

Installation

Note: This documentation explains how to set up the tracing of Go Lambda functions. Ensure that you also have setup the AWS Sensor for Lambda monitoring to ensure the collection of necessary information about versions and runtime metrics that Instana cannot collect from inside the AWS Lambda runtime.

As of v1.23.0 Instana Go in-process sensor automatically detects that a service is running on AWS Lambda and switches to serverless mode. Instead of sending collected traces to the host agent, the in-process sensor submits them directly to Instana serverless acceptor endpoint specified in INSTANA_ENDPOINT_URL environment variable using an agent key defined in INSTANA_AGENT_KEY.

To make sure you're using the latest version of github.com/instana/go-sensor, check the go.mod file in your project or run:

go get github.com/instana/go-sensor@latest

You can download the latest version of Go in-process sensor and update the required version in go.mod.

Configuration

To send collected traces, an AWS Lambda function needs to be provided with two environment variables:

To provide these values to the handler in AWS Console UI, go to the Lambda configuration page:

configuring environment variables

  1. Click your Lambda function box
  2. In within the "Environment Variables" section click "Edit" and add two new variables

There are few optional environment variables that allow changing in-process sensor defaults, such as the list of HTTP headers to collect or a custom service name to use.

Usage

AWS provides github.com/aws/aws-lambda-go package that allows to run Go code on AWS Lambda. To trace Lambda trigger events with Instana and internal and external calls that are made within the handler function, that it needs to be instrumented first. github.com/instana/go-sensor/instrumentation/instalambda provides middleware wrappers for instrumenting the handler code.

To add github.com/instana/go-sensor/instrumentation/instalambda to your project, from the folder containing the go.mod file run:

go get github.com/instana/go-sensor/instrumentation/instalambda

This adds the instrumentation module to your project dependencies list, as well as to the main github.com/instana/go-sensor in-process sensor.

Instrumenting a handler function

A typical AWS Lambda function that is written in Go looks like this:

package main

import (
	"github.com/aws/aws-lambda-go/lambda"
)

func main() {
	lambda.Start(Handle)
}

func Handle() (string, error) {
	// handler code
}

A handler function can take and return up to two parameters, with a limitation that in case the handler function takes two parameters, the first one must implement context.Context.

To instrument a handler function, first create an instrumented handler from it using instalambda.NewHandler() and then pass to lambda.StartHandler(), so the earlier code changes to:

package main

import (
	"github.com/aws/aws-lambda-go/lambda"

	// Import the in-process sensor and instrumentation packages
	instana "github.com/instana/go-sensor"
	"github.com/instana/go-sensor/instrumentation/instalambda"
)

func main() {
	// Initialize the instana.Sensor instance
	sensor := instana.NewSensor("my-lambda-handler")

	// Create an instrumented handler from your handler function
	h := instalambda.NewHandler(Handle, sensor)

    // Pass the handler to the lambda.StartHandler() invoke loop
	lambda.StartHandler(h)
}

func Handle() (string, error) {
	// handler code
}

Instrumenting a lambda.Handler

A typical AWS Lambda handler implemented as lambda.Handler looks like this:

package main

import (
	"github.com/aws/aws-lambda-go/lambda"
)

func main() {
	h := &Handler{
		// handler configuration
	}

	lambda.StartHandler(h, sensor)
}

type Handler struct {
	// ...
}

func (h *Handler) Invoke(ctx context.Context, payload []byte) ([]byte, error) {
	// handler code
}

To instrument such handler, wrap it with instalambda.WrapHandler() and pass to labmda.StartHandler():

package main

import (
	"github.com/aws/aws-lambda-go/lambda"

	// Import the in-process sensor and instrumentation packages
	instana "github.com/instana/go-sensor"
	"github.com/instana/go-sensor/instrumentation/instalambda"
)

func main() {
	// Initialize the instana.Sensor instance
	sensor := instana.NewSensor("my-lambda-handler")

   	h := &Handler{
		// handler configuration
	}

    // Wrap and pass the handler to the lambda.StartHandler() invoke loop
	lambda.StartHandler(instalambda.WrapHandler(h, sensor))
}

type Handler struct {
	// ...
}

func (h *Handler) Invoke(ctx context.Context, payload []byte) ([]byte, error) {
	// handler code
}

Trace Context Propagation

A minimal instrumentation only involves a few changes in your main() function and does not require updating your handler code. However, you might consider adding context.Context to the list of arguments of your handler function. In this case instalambda injects the Lambda trigger event span into it. This span can be retrieved with instana.SpanFromContext() and used as a parent to trace internal and external calls that are made within the handler:

func MyHandler(ctx context.Context) error {
	// Pass the handler context to a subcall to trace its execution
	subCall(ctx)

	// ...

	// Propagate the trace context within an HTTP request to another service monitored with Instana
	// using an instrumented http.Client
	req, err := http.NewRequest("GET", url, nil)
    client := &http.Client{
	    Transport: instana.RoundTripper(sensor, nil),
	}

	client.Do(req.WithContext(ctx))

	// ...
}

func subCall(ctx context.Context) {
	if parent, ok := instana.SpanFromContext(ctx); ok {
		// start a new span, using the Lambda entry span as a parent
		sp = parent.Tracer().StartSpan(/* ... */)
		defer sp.Finish()
	}

	// ...
}