Monitoring Google App Engine

Google App Engine (GAE) is a highly scalable application system on a fully managed serverless platform.

We need to run the Instana agent in the same container where the main application is running.

GAE Custom runtime

The Custom Runtime allows you to include additional components like language interpreters, application servers or additional apps to your environment.

Building a container

The Instana agent Dockerfile is available on GitHub and is an excellent starting point to monitor your apps on within GAE.

Clone the repository

git clone 'https://github.com/instana/instana-agent-docker.git'
cd instana-agent-docker

Modification to support Java

To support running Java based apps, the Dockerfile must be modified and it has to contain:

  • All variables needed by the Instana agent, like your agent key or the host agent endpoint.
  • Instana agent repository configuration
  • Java installation
  • Instana agent configuration files
  • replace java_app_name.jar with your application
  • update main-process script for running Java application
FROM alpine:3.7

ENV LANG=C.UTF-8 \
    INSTANA_AGENT_KEY="" \
    INSTANA_AGENT_ENDPOINT="" \
    INSTANA_AGENT_ENDPOINT_PORT="" \
    INSTANA_AGENT_ZONE="" \
    INSTANA_AGENT_TAGS="" \
    INSTANA_AGENT_HTTP_LISTEN="" \
    INSTANA_AGENT_PROXY_HOST="" \
    INSTANA_AGENT_PROXY_PORT="" \
    INSTANA_AGENT_PROXY_PROTOCOL="" \
    INSTANA_AGENT_PROXY_USER="" \
    INSTANA_AGENT_PROXY_PASSWORD="" \
    INSTANA_AGENT_PROXY_USE_DNS=""

RUN echo "@edge http://dl-cdn.alpinelinux.org/alpine/edge/community" >> /etc/apk/repositories && \
    apk update && \
    apk upgrade && \
    apk add --update-cache --update gomplate@edge bash ca-certificates curl docker@edge inotify-tools && \
    curl -sSL https://packages.instana.io/Instana.rsa -o /etc/apk/keys/instana.rsa.pub && \
    echo "https://_:YOUR_INSTANA_AGENT_HERE@packages.instana.io/agent/apk/generic" >> /etc/apk/repositories && \
    apk update && \
    apk add instana-agent-dynamic && \
    ( /usr/glibc-compat/bin/localedef --force --inputfile POSIX --charmap UTF-8 C.UTF-8 || true ) && \
    echo "export LANG=C.UTF-8" > /etc/profile.d/locale.sh && \
    sed -i '$d' /etc/apk/repositories && \
    rm -rf /tmp/* /var/cache/apk/*

# Install OpenJDK-8
RUN apk update && \
    apk --update add openjdk8-jre

ADD org.ops4j.pax.logging.cfg /root/
ADD org.ops4j.pax.url.mvn.cfg /root/
ADD configuration.yaml /root/
ADD com.instana.agent.main.sender.Backend.cfg.tmpl /root/
ADD mvn-settings.xml.tmpl /root/

# This is the run.sh from the repository renamed to instana-agent
ADD instana-agent /root

# Add the jar file and main-process
ADD java_app_name.jar /root
ADD main-process /root

# The wrapper is used to run both processes in the same container
COPY wrapper.sh /root

WORKDIR /root

ENTRYPOINT ["./wrapper.sh"]

Rename run.sh to instana-agent.

Create a new file called main-process and add this content to it:

java -jar /root/java_app_name.jar &

Create wrapper.sh file:

#!/bin/bash

# Start the first process

./main-process -D
status=$?
if [ $status -ne 0 ]; then
  echo "Failed to start main-process: $status"
  exit $status
fi

# Start the second process
./instana-agent -D
status=$?
if [ $status -ne 0 ]; then
  echo "Failed to start instana-agent: $status"
  exit $status
fi

while sleep 60; do
  ps aux |grep main-process |grep -q -v grep
  PROCESS_1_STATUS=$?
  ps aux |grep instana-agent |grep -q -v grep
  PROCESS_2_STATUS=$?
  # If the greps above find anything, they exit with 0 status
  # If they are not both 0, then something is wrong
  if [ $PROCESS_1_STATUS -ne 0 -o $PROCESS_2_STATUS -ne 0 ]; then
    echo "One of the processes has already exited."
    exit 1
  fi
done

Modification to support NodeJS

The process is similar to Java applications. Instead of Java, install NodeJS and the npm package @instana/collector:

ADD package.json /root
ADD app.js /root/
# Install NodeJS
RUN apk update && \
    apk --update add nodejs nodejs-npm
RUN npm install --save @instana/collector

The main-process file needs instruction how to run your NodeJS app:

node app.js &

Your application also needs to require and initialize the @instana/collector package, for details on that refer to our documentation on Node.js.

Summary

  • The Instana agent key must be placed in the Dockerfile in order to install the host agent
  • main-process is used to start the Java process
  • instana-agent is used to start the Instana agent process (renamed from run.sh)
  • The wrapper is used to start both processes

GAE manifest file

After adding and modifying all necessary files in previous steps we can add a GAE manifest file that describes the GAE instance.

runtime: custom
env: flex

service: gae-java-app

manual_scaling:
  instances: 1

env_variables:
  INSTANA_AGENT_KEY: 'YOUR_INSTANA_AGENT_KEY'
  INSTANA_AGENT_ENDPOINT: 'YOUR_INSTANA_HOST_AGENT_ENDPOINT'
  INSTANA_AGENT_ENDPOINT_PORT: '443'
  INSTANA_AGENT_ZONE: 'gae-java-app'

resources:
  cpu: 4
  memory_gb: 4

health_check:
  enable_health_check: False

skip_files:
- ^\.git/.*$

For NodeJS, Python, and Go based apps, you need to enable tracing by configuring an additional environment variable:

INSTANA_AGENT_HTTP_LISTEN: "*"

Your source code must include sensor for the language that you are monitoring (except Java).

Deployment

Finally, we can deploy this file with running the following command:

gcloud app deploy gae-java-app.yaml

After confirmation you can see the output of the whole process or stream logs from the command line by running:

gcloud app logs tail -s gae-java-app

The Instana agent will discover new instances on the map for every deployed app/instance on Google App Engine Flex. Java

NodeJS