Running Instana agent as a non-root user (public preview)

Edit online

You can run the Instana host agent as a non-root user on systemd-based Linux hosts by using Linux kernel capabilities. This feature improves security by allowing the agent to run with minimum required privileges while it continues to provide full monitoring functions.

Note: This feature is available as a public preview. IBM might change it at any time. Use this feature for evaluation only. Test it thoroughly before you use it in a production environment.

Overview

The non-root agent feature uses Linux kernel capabilities that are applied to the agent runtime during installation. This approach allows the Instana agent to run with an arbitrary user account without requiring elevated privileges, significantly improving the security posture of your monitoring infrastructure.

Important: The new kernel capabilities-based approach provides complete feature parity with root-based installations. It supports all sensors and monitoring features without limitations. This approach is a significant advancement over earlier experimental non-root methods, which had sensor restrictions.

Key benefits

  • Enhanced security: Run the agent with minimum required privileges. Reduce the attack surface.
  • Compliance: Meet security requirements that prohibit running services as the root user.
  • Flexibility: Use custom user and group accounts that align with your organization's security policies.
  • Full functionality: All sensors and monitoring capabilities work without any restrictions. Complete feature parity with root-based installations.

How it works

The non-root agent uses Linux kernel capabilities to grant specific privileges to the agent binary. It does not require full root access. During installation, the installer applies the necessary capabilities to the agent runtime, allowing it to perform all monitoring tasks that typically require elevated permissions.

The agent is managed through systemd service files that are specifically configured for non-root operation. These settings make sure proper startup, shutdown, and service management.

Prerequisites

Before you install the non-root agent, make sure that your system meets the following requirements:

  • Operating system: A systemd-based Linux distribution, for example, RHEL 7 or later, Ubuntu 16.04 or later, Debian 8 or later, or SLES 12 or later.
  • systemd: Version 219 or later.
  • Linux kernel: Version 3.10 or later with support for capabilities.
  • Package manager: An RPM-based package manager (yum or dnf) or a Debian-based package manager (apt).
  • Permissions: Root or sudo access for initial installation (the agent runs as a non-root after installation).

To verify the systemd version:

systemctl --version

To verify the Linux kernel version:

uname -r

Environment variables

The following environment variables control the non-root agent feature:

Variable Description Default Required
INSTANA_AGENT_NON_ROOT Enable the non-root agent feature false No (automatically set to true if both user and group are specified)
INSTANA_AGENT_USER (optional) Specifies the user account that runs the agent instanaagent (used when INSTANA_AGENT_NON_ROOT=true) No
INSTANA_AGENT_GROUP (optional) Specifies the group account that runs the agent instanaagent (used when INSTANA_AGENT_NON_ROOT=true) No

Set these environment variables before installation. The Debian and RPM package preinstall scripts process these variables during installation.

Custom user and group installation

If you need to use a specific user and group account, you can provide custom values during installation.

  1. Create the user and group before installation if they do not exist: 
    sudo groupadd -r myagentgroup
sudo useradd -r -g myagentgroup -s /sbin/nologin -c "Instana Agent User" myagentuser
  2. Set the custom user and group environment variables: 
    export INSTANA_AGENT_USER=myagentuser
export INSTANA_AGENT_GROUP=myagentgroup
Note: When you set both INSTANA_AGENT_USER and INSTANA_AGENT_GROUP, the installer automatically enables the INSTANA_AGENT_NON_ROOT flag.

Installation

By default, the non-root agent creates and uses a system user and group named instanaagent. This approach is recommended for most deployments.

Note: For more information about IBM MQ monitoring with non-root agent, see the limitations section for further installation details.
  1. Before installation, set the INSTANA_AGENT_NON_ROOT environment variable to enable the non-root feature: 
    export INSTANA_AGENT_NON_ROOT=true
  2. Install the agent by using one of the standard methods that are described in the main Linux installation sections:

Limitations

Review the following limitations when you use the non-root agent:

Systemd requirement

The non-root agent feature is only available on systemd-based Linux distributions. Systems that use other init systems, such as SysV init or Upstart, are not supported.

Linux-only support

This feature is only available for Linux hosts. Other operating systems, such as Windows, macOS, and Unix variants are not supported for non-root operation.

Kubernetes and Red Hat OpenShift

The non-root agent does not yet support Kubernetes and Red Hat OpenShift deployments. Installing the agent on Kubernetes or Red Hat OpenShift Container Platform (OCP) still requires elevated permissions.

IBM MQ requirement

For IBM MQ monitoring with the non-root agent, you need to verify that the user and group name for the agent is a name with 12 characters or shorter. The default user and group name, instanaagent, is sufficient. You can also use any user and group name that is shorter or equal a 12 character length.

If you use a custom user and group name, make sure to set the appropriate environment variables before you run the installer. For more information, see the custom user and group installation section.

PHP monitoring

The non-root agent does not support monitoring:
  • PHP runtimes in rootless Podman environments.
  • PHP-FPM runtimes that use TCP listen addresses in pm.status_listen or listen PHP-FPM configuration directives.

Upgrade considerations

In the initial public preview release, you cannot upgrade a root-based agent to a non-root agent by using the standard upgrade process. Uninstall the existing agent. Then, reinstall the agent and enable the non-root configuration.
Note: You can upgrade between non-root agent versions by using the standard package-manager upgrade process.

Externally provided Java runtime

An externally provided Java runtime for the agent is not supported when you run the agent as a non-root user. Only the Java runtime from the Instana agent install-package is supported.

Complete sensor and feature support

The non-root agent that uses kernel capabilities supports all Instana sensors and monitoring features. This support includes:
  • All infrastructure monitoring sensors
  • Application performance monitoring (APM) for all supported technologies
  • Process monitoring and tracing

This kernel capabilities-based approach achieves complete feature parity with root-based agent installations. Unlike earlier experimental non-root approaches that had sensor restrictions, this new implementation provides full monitoring functions.

Kernel capabilities reference

The non-root agent requires specific Linux kernel capabilities to do its monitoring functions. During installation, the systemd service configuration applies these capabilities automatically to the agent binary. The following table provides a comprehensive reference of all required capabilities and their purposes:

Capability Purpose Details
CAP_AUDIT_WRITE Audit logging Allows sudo and other privileged helpers to write audit events to the kernel audit subsystem. Without this capability, sudo still works but logs the message, "sudo: unable to send audit message: Operation not permitted".
CAP_SYS_ADMIN Namespace operations Required for setns() on most namespaces (net, ipc, uts, pid, and cgroup). Enables nsenter to access host namespaces for comprehensive system monitoring.
CAP_SYS_CHROOT Mount namespace access Required to enter mount namespaces with setns(--mount). Allows the agent to inspect file system hierarchies across different mount namespaces.
CAP_SYS_PTRACE Process inspection Required to inspect and interact with processes that other users own. Needed for cross-user JVM attach and reading foreign /proc/pid/ directories for comprehensive process monitoring.
CAP_SYS_RESOURCE Resource limits and eBPF Required to raising resource limits (setrlimit). Required by eBPF profiling to increase RLIMIT_MEMLOCK so BPF maps and perf ring buffers can be pinned in memory.
CAP_NET_ADMIN Network monitoring Used by network discovery and host network-inspection features. Enables comprehensive network topology mapping and configuration analysis.
CAP_NET_RAW Raw socket operations Needed for raw sockets (for example, ICMP or ping, and packet-level operations). Essential for network diagnostics and low-level network monitoring.
CAP_DAC_OVERRIDE DAC bypass (write and execute) Allows bypassing DAC write and execute checks. Enables access to Unix domain sockets or files that might use mode 0600. Allows access to container sockets for container monitoring.
CAP_DAC_READ_SEARCH Discretionary access control (DAC) bypass (read) Bypasses DAC for read and search operations. Required to read other users' /proc/pid/ directories and cgroup information without permission restrictions.
CAP_FOWNER Filesystem ownership bypass Required for cleanup tasks where the Instana agent removes stale IBM J9 attach directories (for example, /tmp/.com_ibm_tools_attach/<pid>) that are left behind by other JVMs and owned by different users.
CAP_KILL Signal operations Required for JVM attach operations. For example, Byte Buddy sends a signal to the target JVM. Needed to bypass signal permission checks across user boundaries for application monitoring.
CAP_SETUID or CAP_SETGID User and group switching Required to change the effective UID or GID of a process. If present in both CapabilityBoundingSet and AmbientCapabilities, the Instana Agent JVM can perform in-process user switches. If present only in CapabilityBoundingSet (but not in AmbientCapabilities), the JVM does not have these capabilities, but setuid-root helpers such as sudo can still acquire them. Without these caps in the bounding set, sudo cannot perform its internal setresuid() or setresgid() transitions and fails with an "Operation not permitted" error.
CAP_SYSLOG EBPF kernel symbols Allows reading kernel symbol addresses from /proc/kallsyms/ when kptr_restrict is enabled. Required by eBPF profiling to resolve kernel symbols and stack traces for kprobes and kernel-level profiling. Without this capability, kernel addresses may be hidden, preventing proper symbol resolution in profiling output.

The systemd service file configures these capabilities. The agent process inherits them automatically at startup. The capabilities are set with both the permitted and effective flags to make sure that they are available to the agent runtime.

Run the following command to verify the capabilities assigned to the running agent process:

grep CapEff /proc/$(pgrep -f 'karaf.home=.*instana')/status

Run the following command to decode the capability hex values:

capsh --decode=<hex_value>
Important: Do not manually modify these capabilities. Removing a required capability causes monitoring features to fail.