Monitoring Containerd
After you install the Instana host agent, the Containerd sensor is automatically installed.
You can view metrics that are related to Containerd in the Instana UI after you configure the Containerd sensor as outlined in the Configuring section.
Supported information
Supported operating systems
The supported operating systems of the Containerd sensor are consistent with host agents requirements, which can be checked in the Supported operating systems section of each host agent, such as Supported operating systems for Unix.
Supported environments
If the environment runs Docker or Garden containers, then the Containerd sensor is automatically disabled to prevent multiple sensors from unnecessarily monitoring the same containers.
Supported containers
All containers that run the Containerd container runtime are supported except for the pause containers.
The pause container is a container that holds the network namespace for the pod. Kubernetes creates pause containers to acquire the IP address of respective pod and set up the network namespace for all other containers that join that pod.
The pause containers are not supported for the following reasons:
- Monitoring pause containers does not bring much information on the infrastructure level because they act as sidecar helper containers.
- When the pause containers are included, it doubles the number of monitored containers in an environment and therefore increases the Instana monitoring costs.
Configuring
The Containerd sensor requires the crictl utility to collect Kubernetes labels that are associated with the container and locate Container logs. The Kubernetes labels that are associated with the container are displayed
in the Kubernetes Labels tab in the Instana UI.
To enable the collection of these labels, you must install the crictl command-line tool in the host server where the Instana host agent is installed. Then, add the path of the crictl binary to the $PATH environment variable for the user under which the host agent is running.
If the crictl command-line tool is not installed, the Containerd sensor's performance metric collection for CPU and memory usage can work, but the Kubernetes labels that are associated with the container can't be displayed in the
Kubernetes Labels tab in the Instana UI, and the container log is not collected when you enable the feature.
Configuring the Containerd sensor
You can configure the Containerd sensor in the agent configuration file as shown in the following example:
com.instana.plugin.containerd:
enabled: true
logs:
enabled: true
sendInterval: 60
maxBufferSize: 16777216
unixSockets:
- '/path/to/containerd.sock'
| Parameter | Description | Mandatory/Optional | Value range |
|---|---|---|---|
| com.instana.plugin.containerd.enabled | The flag to enable the Containerd sensor. | Optional | false or true.Default: true |
| com.instana.plugin.containerd.logs.enabled |
Available since: Containerd sensor 1.0.30. The flag to enable log collection for the Containerd sensor. |
Optional | true or false. Default: false |
| com.instana.plugin.containerd.logs.sendInterval |
Available since: Containerd sensor 1.0.30. The maximum duration (in seconds) that container logs remain cached in the log-sensor before they are uploaded to the backend. |
Optional | 1 to 600. Default: 60 |
| com.instana.plugin.containerd.logs.maxBufferSize |
Available since: Containerd sensor 1.0.30. The maximum internal log buffer size (in bytes). If internal log buffer contents exceed the size, then the log-sensor flushes the contents to the Instana backend. |
Optional | 1000 to 32000000. Default: 16777216 (16 MB) |
| com.instana.plugin.containerd.unixSockets |
Available since: Containerd sensor 1.0.30. List of fully qualified paths to containerd.sock files. The sensor has automatic socket file discovery that searches for the containerd.sock file in /run/ and /var/run/ directories. If a different location is configured for the socket, then the fully qualified path needs to be added. |
Optional | List of fully qualified file paths. |
Enabling metrics collection
The Containerd sensor with ctr versions after 1.2.0 can automatically collect metrics by using the ctr task metrics command. In this case, you don't need to manually enable the metrics collection.
The Containerd sensor with ctr 1.2.0 and earlier versions does not support the metrics command. If you are using ctr 1.2.0 or earlier versions, the sensor will attempt
to query metrics from the Prometheus endpoint. To enable the Prometheus endpoint, you need to specify the "metrics" address "127.0.0.1:1338" in the /etc/containerd/config.toml configuration file
and then restart the containerd.service.
[metrics]
address = "127.0.0.1:1338"
Viewing metrics
To view the metrics, complete the following steps:
- In the sidebar of the Instana UI, select Infrastructure.
- Click a host that has a Containerd container.
You can see a host dashboard with all the collected metrics and monitored processes.
Viewing container details
To fetch the container details, click Get Container Info.
The following information is displayed:
- Container ID
- Image
- Created at
- Updated at
- Containerd namespace
- Labels
Performance metrics
| Metric | Description | Granularity |
|---|---|---|
| CPU usage | The metrics include total CPU usage, kernel CPU usage, user CPU usage, and normalized CPU usage | 10 seconds |
| CPU throttling | Throttling time and count | 10 seconds |
| Memory usage | The metrics include total memory usage in bytes, resident set size in bytes, cache usage in bytes, total memory usage in percentage, and working set usage in percentage | 10 seconds |
| Memory active | The amount of anonymous and cache active memory | 10 seconds |
| Memory inactive | The amount of anonymous and cache inactive memory | 10 seconds |
| Block IO | The amount of data, in bytes, that the monitored container writes to and reads from the block device | 10 seconds |
Derived performance metrics
You can view the following derived metrics on the Instana UI:
- CPU Total Normalized
- Memory Usage %
- Memory Working Set Usage %
Total Normalized CPU Usage
The CPU Total Normalized metric is the normalized CPU usage of the container in percentage. The value can range from 0% to 100%. The metric is computed by using the following formula:
total_normalized_cpu_usage_percentage = cpu_total_usage / cpu_limit
The cpu_limit value depends on the configuration of the container that is deployed. The value of the total_normalized_cpu_usage_percentage metric is not known if cpu_limit is not specified in the configuration
of the container that is deployed.
Known limitations
The CPU Total Normalized metric is supported only on systems that are using Control Group V1 (cgroupv1).
Memory Usage
The Memory Usage % metric is the memory usage in percentage, including resident set size and file system cache usage. The metric is computed by using the following formula:
metric_usage_percentage = memory_used / memory_limit
The value of the memory_limit metric depends on the configuration of the container that is deployed. The value of the memory_used metric is not known if memory_limit is not specified in the configuration
of the container that is deployed.
Memory Working Set Usage
The Memory Working Set Usage % is the memory usage in percentage excluding the file system cache. The metric is computed by using the following formula:
memory_working_set_usage_percentage = (memory_used - total_inactive_file) / memory_limit
The memory_limit value depends on the container's deployment configuration. The memory_working_set_usage_percentage is not provided when memory_limit is not specified in container's deployment configuration.
Known limitations
The Memory Working Set Usage % metric is supported only on systems that are using Control Group V1 (cgroupv1).
Viewing container logs
Containerd log collection is supported for Containerd sensor 1.0.30 and later.
When this feature is enabled, the Containerd sensor collects only the container logs that are written to a file because Instana currently supports only the default logging driver, which writes log entries to a file.
For example, in Kubernetes deployments, Containerd logs might be written to files in the /var/log/pods/{namespace}_{podName}_{podUID}/{containerName} directory.
You can monitor the collected logs by clicking Analytics > Logs in the Instana UI and filter logs by using the "Container Snapshot Id" filter, which can filter both Docker and Containerd log messages.