Monitoring AWS Fargate
Instana supports AWS Elastic Container Service (ECS) with two launch types, Fargate and Elastic Compute Cloud (EC2).
To monitor the containerized applications that are running on AWS Fargate, install Fargate collectors as described in the Setting up Instana Fargate collectors section. You don't need to install the host agent.
For using AWS ECS with the launch type EC2, see Installing the host agent in AWS ECS on EC2.
AWS Fargate on Elastic Kubernetes Services (EKS) is not supported.
- Setting up Instana Fargate collectors
- Configuring task definition
Setting up Instana Fargate collectors
To monitor applications that are running on AWS Fargate, Instana uses the following Fargate collectors:
Set up these collectors for Instana to start monitoring.
Node.js
The Node.js Fargate collector is delivered over a Docker agent image layer that contains the collector files and an initialization script.
-
If your application is developed by using CommonJS, set up the Node.js Fargate collector by following the steps in the CommonJS section.
-
If your application is developed in ECMAScript modules, set up the Node.js Fargate collector by following the steps in the ECMAScript modules section.
CommonJS
-
Add the following lines to your
Dockerfile
anywhere after the lastFROM
clause:COPY --from=icr.io/instana/aws-fargate-nodejs:latest /instana /instana RUN /instana/setup.sh ENV NODE_OPTIONS="--require /instana/node_modules/@instana/aws-fargate"
-
Build the container and push it to the image repository of your choice.
-
Complete the configuration steps that are described in the Configuring task definition.
ECMAScript modules
- Add the following lines to your
Dockerfile
anywhere after the lastFROM
clause:
-
For Node.js 18.19.0 and later, add the following commands:
COPY --from=icr.io/instana/aws-fargate-nodejs:latest /instana /instana RUN /instana/setup.sh ENV NODE_OPTIONS="--import=/instana/node_modules/@instana/aws-fargate/esm-register.mjs"
-
For Node.js versions earlier to 18.19.0, add the following commands:
COPY --from=icr.io/instana/aws-fargate-nodejs:latest /instana /instana RUN /instana/setup.sh ENV NODE_OPTIONS="--experimental-loader=/instana/node_modules/@instana/aws-fargate/esm-loader.mjs"
-
Build the container and push it to the image repository of your choice.
-
Complete the configuration steps that are described in the Configuring task definition.
Notes:
- If you are using multi-stage builds (that is, if you have multiple
FROM
clauses in yourDockerfile
), add these lines after the lastFROM
clause of yourDockerfile
. - Auto-profiling of Node.js processes in AWS Fargate is not supported.
Versioning
The icr.io/instana/aws-fargate-nodejs:latest
Docker image uses the latest available image version, regardless of the current major version. For example, if a major version 4 is released and currently your image version is 3,
then the image is automatically updated to version 4 on the next rebuild if you use the tag :latest
.
If you do not want the latest image version that is available, use a major version of the image like icr.io/instana/aws-fargate-nodejs:2
(version 2) and manually update to the next major version.
For more information, see Node.js versioning.
Build dependencies and native add-ons
If you use a minimal base image for your Node.js containers, which does not have the required packages to build native Node.js add-ons, you can see the following warning in your Docker build:
gyp ERR! ...some error message from trying to rebuild native add-ons, for example:
gyp ERR! stack Error: Could not find any Python installation to use
...
Warning: Rebuilding native add-ons for @instana/fargate failed. Monitoring the Fargate tasks will work nonetheless, but you will miss some Node.js metrics (GC metrics, event loop metrics, ...). See https://www.instana.com/docs/ecosystem/aws-fargate/#build-dependencies-and-native-add-ons for details."
The log message implies that Instana monitors your Node.js Fargate task, but it might miss certain metrics. To avoid metrics loss, install the missing operating system packages, at least temporarily, during the build. These packages can be excluded in the final image. The type of packages and how to install them depends on your distribution.
See the following example Dockerfile snippet for Linux Alpine:
COPY --from=icr.io/instana/aws-fargate-nodejs:latest /instana /instana
RUN apk add --no-cache --virtual .gyp \
build-base \
python \
&& /instana/setup.sh \
&& apk del .gyp python
ENV NODE_OPTIONS="--require /instana/node_modules/@instana/aws-fargate"
See the following example for Debian-based distributions (including Ubuntu):
COPY --from=icr.io/instana/aws-fargate-nodejs:latest /instana /instana
RUN apt-get update \
&& apt-get install build-essential \
&& /instana/setup.sh \
&& apt-get remove -y build-essential
ENV NODE_OPTIONS="--require /instana/node_modules/@instana/aws-fargate"
For distributions that use the yum
package manager (like CentOS and RHEL), run the following command to install the required packages:
RUN yum groupinstall "Development Tools"
Getting the Node.js layer from containers.instana.io
The Docker layer is available publicly on the IBM container registry. You can obtain the Node.js Instana Fargate image layer from it. Alternatively, you can also obtain the image layer from the containers.instana.io
registry.
For example, if you also use Instana Fargate support with other runtimes (like Java®), for which the image layer is only available on the authenticated registry containers.instana.io
, and you want consistency, you can get
the Node.js image layer from containers.instana.io
instead of the IBM container registry.
To get the Node.js image layer from the containers.instana.io
registry, complete the following steps:
-
Because the
containers.instana.io
registry requires authentication, run the following command before you use it in your Docker build:docker login -u _ -p <your-agent-key> containers.instana.io
Replace the placeholder
<your-agent-key>
with your Instana agent key on the system where you build your Docker image. -
Add the following lines to your
Dockerfile
anywhere after the lastFROM
clause:COPY --from=containers.instana.io/instana/release/aws/fargate/nodejs:latest /instana /instana RUN /instana/setup.sh ENV NODE_OPTIONS="--require /instana/node_modules/@instana/aws-fargate"
.NET Core
The way that you make .NET Core tracing available in your Fargate containers is the same as for any other .NET Core containerized application. You have two options:
- Install NuGet packages in your application during development
- Install NuGet packages by using Dockerfile
Install NuGet packages during development
To install NuGet packages during development, you must add the NuGet packages for the rewriter and the managed tracing extensions to your project at design-time, or complete the following steps as part of your build when you prepare your container-image:
Install the rewriter
Depending on the OS that is backing your container, you must choose one of the following rewriter packages that is built from native code and targets specific system libraries:
- For container images based on Alpine Linux, use the Alpine Rewriter Package.
- For any other Linux, use the Linux Rewriter Package.
Install the tracing package
The packages also install the most recent version of the tracing package that was available at the time when the rewriter package is released. However, add the Instana tracing package for .NET Core to get more control over the version of the tracing package that your application uses.
Configure the .NET Core task definition
Set the following environment variables by following the configuration steps that are described in Configuring task definition:
CORECLR_ENABLE_PROFILING=1
This setting enables profiling of .NET Core.CORECLR_PROFILER={cf0d821e-299b-5307-a3d8-b283c03916dd}
This setting sets theclsid
of the profiler to be used.CORECLR_PROFILER_PATH=/app/instana_tracing/CoreProfiler.so
This setting points to the library that contains the profiler, which assumesapp
to be the path of your application.DOTNET_STARTUP_HOOKS=/app/Instana.Tracing.Core.dll
This setting sets metrics collection and support of self-contained .NET Core applications.
Install NuGet packages in the Docker build
To install NuGet packages in a Docker container, complete the following steps:
-
If you use a container image based on Alpine Linux, replace the first line with the appropriate NuGet package postfix that uses
Alpine
as in the following example. Otherwise, useLinux
.# after building the project, right before publishing, add the nuget-packages RUN dotnet add <project_name>.csproj package Instana.Tracing.Core.Rewriter.<Linux|Alpine> --version <min-version 1.0.56> RUN dotnet add <project_name>.csproj package Instana.Tracing.Core --version <min-version 1.1.46> RUN dotnet publish -c Release -o out # where the files will end up WORKDIR /app COPY --from=publish /src/out . # set the env-vars as discussed above ENV CORECLR_ENABLE_PROFILING=1 ENV CORECLR_PROFILER={cf0d821e-299b-5307-a3d8-b283c03916dd} ENV CORECLR_PROFILER_PATH=/app/instana_tracing/CoreProfiler.so ENV DOTNET_STARTUP_HOOKS=/app/Instana.Tracing.Core.dll ... # Other stuff in your Docker image
-
Follow the configuration steps that are described in Configuring task definition.
Go
The Go Fargate collector is included in github.com/instana/go-sensor
1.14.0 and later versions. This collector does not require extra configuration apart from
setting up the environment variables that are described in the Configuring task definition section. The tracer detects that the application is running on AWS Fargate and activates the Fargate collector
automatically.
Ruby
The Ruby Fargate collector is included in github.com/instana/ruby-sensor
1.197.0 and later versions. This collector uses the standard configuration that is described in the
Configuring task definition section. The tracer detects that the application is running on AWS Fargate and activates the Fargate collector automatically.
Java
The Java® Fargate collector is delivered over a Docker agent image that contains the collector files.
The Docker layer is available in the containers.instana.io
registry. To get the Java image layer from the Docker registry, complete the following steps:
-
Because the Docker registry requires authentication, log in to the registry with your agent key before you use the registry in your Docker build:
docker login -u _ -p <your-agent-key> containers.instana.io
Replace the placeholder
<your-agent-key>
with your actual Instana agent key on the system where you build your Docker image. -
Add the following lines to your
Dockerfile
anywhere after the lastFROM
clause:COPY --from=containers.instana.io/instana/release/aws/fargate/jvm:latest /instana /instana ENV JAVA_TOOL_OPTIONS="-javaagent:/instana/instana-fargate-collector.jar"
-
Build the container and push it to the image repository of your choice.
-
Follow the configuration steps that are described in the section Configuring task definition.
Notes:
-
If you are using multi-stage builds (that is, if you have multiple
FROM
clauses in yourDockerfile
), add the following lines in yourDockerfile
after the lastFROM
clause. -
Auto-profiling of Java processes in AWS Fargate is not supported.
Secrets
The Java Fargate collector supports stripping of secrets much like the host agent.
You can specify patterns for secrets, that is, data is filtered from tracing and process details. The data that is treated as secrets doesn't reach Instana SaaS for processing and is, therefore, not available for analysis in the Instana UI or retrieval by using API.
Secrets are defined by the environment variable INSTANA_SECRETS
by using the following format:
<matcher>:<secret>[,<secret>]
matcher
can be one of the following values with the respective semantic for the secrets provided:
equals-ignore-case, equals, contains-ignore-case, contains, regex
If no secrets are configured, a default secret is used. The following setting is enabled to obtain the secret:
INSTANA_SECRETS=contains-ignore-case:key,password,secret
If a key matches an entry from the list of secret, the value is redacted and not sent to the Instana backend.
Secrets at infrastructure and platform level are process environment variables.
Secrets at run time:
- JDBC connection strings
- HTTP query parameters, such as
https://my.domain/accounts/status?account=<secret_1>&user=<secret_2>
(for all supported runtimes, see the following support matrix) - HTTP matrix path parameters, such as
https://my.domain/accounts/account=<secret_1>;user=<secret_1>/status
Note:
-
Instana does not support treating arbitrary segments in a URL path as secrets. For example, if the URL is structured as
https://my.domain/accounts/<secret>/status
. The overhead of running, for example, regular expressions on URL paths to discard segments is prohibitive. -
Instana does not capture SQL parameters from stored procedures. Similarly, for arbitrary path segments, literals from SQL queries are not stripped; use parametrized queries.
Java Trace SDK
The Java Fargate collector supports the Java Trace SDK, similar to the agent. However, the Java Fargate collector is activated by the environment variable INSTANA_INSTRUMENTATION_SDK_PACKAGES
and not by the agent configuration.
Detailed information about the usage of the Java Trace SDK is available on its documentation page.
INSTANA_INSTRUMENTATION_SDK_PACKAGES
can contain a list of comma-separated Java package names for which the SDK is activated. For example,
INSTANA_INSTRUMENTATION_SDK_PACKAGES=my.first.package,my.second.package
Python
The Python Fargate collector is included in the standard Instana Python Package 1.25.0 and later versions. The collector does not require extra configuration apart from setting up the environment variables that are described in the Configuring task definition section. The package detects that the application is running on AWS Fargate and activates the Fargate collector automatically.
Configuring task definition
Notes:
-
The instructions for configuring task definition are common to setting up all Instana collectors on AWS Fargate.
-
Before you configure the task definition, ensure that the container images are built according to the instructions specific to the runtime that your application uses.
-
If your application is based on .NET Core and you are installing the Instana NuGet packages at development time as described in Install NuGet Packages during Development, ensure that you set the environment variables that are listed in Configure the .NET Core task definition in your task definition.
All Instana collectors on AWS Fargate need a common set of configuration values to connect and authenticate with the Instana backend. To configure your task definition, complete the following steps:
-
Log in to the AWS console, and go to ECS.
-
In the navigation panel, click Task Definitions.
-
In the task definition configuration page, click Create new revision to create a revision of the task definition:
-
Go to the container definition of your Node.js-based container:
-
Add the following two environment variables to the container specification:
INSTANA_ENDPOINT_URL
: This environment variable is your serverless monitoring endpoint. Make sure that you use the correct value for your region that starts withhttps://serverless-
.INSTANA_AGENT_KEY
: This environment variable is your agent key.
You can also obtain the correct values for these environment variables by going to your Instana UI and clicking More > Agents > Installing Instana Agents > Platform: "AWS" > Technology AWS Fargate.
For more information about environment-based configurations that AWS Fargate collectors support, see General Reference - Environment Variables.