Installing the host agent on Linux

You can install the Instana host agent on Linux in an online or offline (air-gapped) environment.

In an online environment, to have the installation steps completed automatically for you, use the one-liner installation technique on supported Linux distributions.

Alternatively, in an online or offline (air-gapped) environment, you can download the agent package manually and then either install the package manually or create a repository and instruct your package manager to install the package for you. You can also install and run the agent from a tarball archive.

In situations where it is undesirable or impossible to make the agent communicate with the Instana hosted agent repositories, you can use a local mirror instead. You can configure the Instana host agent to use the agent repository as the mirror, configure a different mirror, or can change the agent repository location. For more information about using your own agent repository, see Setting up agent repositories for dynamic host agents.

Supported operating systems

The Instana host agent is supported on the following Linux operating systems and distributions:

  • Ubuntu Linux 14.04 (trusty)
  • Ubuntu Linux 16.04 (xenial)
  • Ubuntu Linux 18.04 (beaver)
  • Ubuntu Linux 20.04 (focal)
  • Ubuntu Linux 22.04 (jammy)
  • CentOS 6 [1]
  • CentOS 7
  • CentOS 8
  • CentOS 9
  • Debian 9 (stretch)
  • Debian 10 (buster)
  • Debian 11 (bullseye)
  • Suse Linux Enterprise Server (SLES) 12 [2]
  • Suse Linux Enterprise Server (SLES) 15
  • Red Hat Enterprise Linux (RHEL) 6 [3]
  • Red Hat Enterprise Linux (RHEL) 7
  • Red Hat Enterprise Linux (RHEL) 8
  • Red Hat Enterprise Linux (RHEL) 9
  • Oracle Enterprise Linux 7
  • Oracle Enterprise Linux 8
  • Oracle Enterprise Linux 9
  • Amazon Linux 1
  • Amazon Linux 2
  • Alma Linux 8
  • Alma Linux 9
  • z/Linux, Linux on Z (s390x, 64 bit) [3:1]
  • z/OS (OS/390) - (s390x) - Supported WebSphere and Liberty tracing
  • Kylin Linux Advanced Server v10 (x86_64 & aarch64) [4]

Supported platform architectures

The Linux operating systems and distributions are supported on the following platform architectures:

Name Bitness Vendor
x86 32 bit Intel/AMD
x64, amd64 64 bit Intel/AMD
arm 32 bit ARM
arm64, aarch64 64 bit ARM
ppc 32 bit IBM Power
ppc 64 bit IBM Power
ppc64le 64 bit IBM Power, little endian
s390x 64 bit IBM z/Architecture

Automatic installation (one-liner)

The agent one-liner is a powerful script that performs automated installations of the host agent on Linux-based operating systems.

Supported Distributions

The one-liner supports the following Linux distributions:

  • Debian derivatives (apt package manager)
  • Red Hat derivatives (yum package manager)
  • SUSE derivatives (zypp package manager)

Using the one-liner

You need root user privileges to install the Instana host agent.

To use the one-liner script to install the host agent and then run the agent, complete the following steps:

  1. Sign in to the Instana UI, and then select an option to display the agent catalog; for example, on the home page, click Deploy Agent located at the top right corner.

    If you are starting a new trial instance of Instana, the agent catalog is displayed with a prompt to select a host agent to install.

  2. Click the tile Linux - Automatic Installation (One-liner)

  3. Select the packaging option to install a dynamic or static agent, and the runtime options that you want to use. If you change options, then the script parameters are updated.

    The script is pre-filled with your agent key and host agent endpoint.

    For example, the UI provides the following script to install the agent with the Dynamic packaging option, and with the Azul Zulu 1.8 and Interactive runtime options:

    curl -o setup_agent.sh https://setup.instana.io/agent && chmod 700 ./setup_agent.sh && sudo ./setup_agent.sh -a <your_agent_key> -d <your_agent_key> -t dynamic -e <host-agent-endpoint>
    
  4. (Optional) Select the option to install and start the host agent as a service (only supported for SystemD-based systems).

  5. Copy and then run script to install the host agent.

  6. To start the Instana host agent, run one of the following commands:

    • If your operating system uses systemd as the init system (and you have not already started the agent as a service), run the systemctl start instana-agent.service command.
    • If your operating system uses SysVinit as the init system, run the service instana-agent start command.

Additional parameters

The one-liner command accepts the following parameters:

Parameter Description
-a = (required) The agent key.
-d = (optional) The Instana download key. If you are using a self-hosted (on-premises) Instana deployment, the key is provided for you by Instana.
-e = (required) The host agent endpoint.
-m = (optional) Sets the Instana agent mode, apm (default), infra, or aws.
-t = (optional) The agent flavor, dynamic (default) or static.
-j = (optional) Selects Eclipse OpenJ9 11 as the bundled Java runtime.
-y = (optional) Noninteractive prompt. Specify it if the setup is being performed without an interactive shell.
-s = (optional) Start the instana-agent service, and enable it to start at boot time. This option works only for systems that is running systemd.
-g = (optional, needed if -b is set) The specification of the remote URL for the Git-based configuration management capability of the host agent.
-b = (optional, needed if -g is set) The name of the remote branch to track for the Git-based configuration management capability of the host agent.
-u = (optional, needed if -p is set) The username for basic authentication if you are using HTTP-based remotes for the Git-based configuration management capability of the host agent.
-p = (optional) The password for basic authentication if you are using HTTP-based remotes for the Git-based configuration management capability of the host agent.

When you download and run one-liner.sh script, it adds the public systems package repository of Instana to your system by creating a repository file. This file adds Instana authenticated repositories to the machine's installation sources.

Potential problems during installation

Debian-based

With Debian-based derivatives, you might experience the following output when you issue the installation. In this case, install the package apt-transport-https because Apt doesn't currently pull fonts from HTTPS servers.

  Setting up Instana APT repository
  Importing Instana GPG key
  Updating apt metadata ...
  E: The method driver /usr/lib/apt/methods/https could not be found.
  APT repository metadata update failed

SUSE and SLES

If the SUSE Linux Enterprise Server 12 gives you the following output, you need to update your openssl to properly receive artifacts through modern HTTPS connections:

  Setting up Instana agent for GNU/Linux
  Setting up Instana zypper repository
  Updating zypper metadata ...
  Installing Instana agent ...
  Error building the cache:
  [instana-agent|https://_@packages.instana.io/agent/generic/x86_64] Valid metadata not found at specified URL
  Some of the repositories have not been refreshed because of an error.
  No provider of 'instana-agent-static' found.
  Instana agent package install failed

CentOS

On CentOS 6 based systems, outdated libcurl versions can cause this log output:

Setting up Instana agent for GNU/Linux
Setting up Instana YUM repository
Updating YUM metadata ...
YUM repository metadata update failed

RHEL

Enabled FIPS (Federal Information Processing Standard) or SELinux can lead to the following error during one-liner installation:

Error unpacking rpm package instana-agent-dynamic-20210630-0948.x86_64
Error: Transaction failed

Manual installation of instana-agent packages fails in the following setup:

sudo rpm -ivh instana-agent-dynamic-20210713-1352.x86_64.rpm
...
error: unpacking of archive failed on file /etc/init.d/instana-agent;60eda3b3: cpio: Digest mismatch
error: instana-agent-dynamic-20210713-1352.x86_64: install failed

A possible workaround for manual installation is to skip verification with the following command:

sudo rpm -ivh --nodigest --nofiledigest instana-agent-dynamic-20210713-1352.x86_64.rpm

Manual installation

For manual installation, you can download Instana agent packages manually. You can then install packages manually or create a repository and instruct your package manager to install packages for you.

Downloading the agent package

To download an Instana agent package, complete the following steps:

  1. Sign in to the Instana UI, and then select an option to display the agent catalog; for example, on the home page, click Deploy Agent.

    If you are starting a new trial instance of Instana, the agent catalog is displayed with a prompt to select a host agent to install.

  2. Click the tile Linux - Packages (DEB, RPM)

  3. Copy and paste the URL for the Instana Agent Package Downloads page into your web browser: https://_:<your_agent_key>@packages.instana.io/agent/download.

    The URL is pre-filled with your agent key.

  4. Download an agent package for DEB or RPM.

Next, either instruct your package manager to install packages for your Linux distribution or install the agent package manually.

Debian derivatives (apt package manager)

To prepare Apt with resources for the Instana agent packages, create the following files:

# /etc/apt/sources.list.d/instana-agent.list
deb [arch=amd64] https://packages.instana.io/agent generic main
# /etc/apt/auth.conf.d/instana-packages.conf
machine packages.instana.io
login _
password <your_agent_key>

Alternatively, you might hardcode the authentication in the URL in /etc/apt/sources.list.d/instana-agent.list, but it causes a mild security risk and apt issues warnings that are shown whenever you install or update Instana packages.

To add the Instana GPG key to APT, run the following command:

wget -qO - https://packages.instana.io/Instana.gpg | sudo apt-key add -

RPM-based distributions (yum package manager)

Copy the following code snippet into your yum sources (/etc/yum.repos.d/Instana-Agent.repo):

[instana-agent]
name=Instana
baseurl=https://_:<your_agent_key>@packages.instana.io/agent/generic/x86_64
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://packages.instana.io/Instana.gpg
priority=5
sslverify=1

Replace <your_agent_key> with your agent key.

If you want to enable GPG package verification, some distributions require that you import the key into the package manager's key ring beforehand.

To import the key, run the following command:

rpm --import https://packages.instana.io/Instana.gpg

openSUSE/SLES (zypp package manager)

For openSUSE and SUSE Linux Enterprise Server, the path is /etc/zypp/yum.repos.d/Instana-Agent.repo, but the contents can remain the same as RPM-based distribution.

Preparing the agent

After you refresh the package manager's sources, you need to configure the agent with the host agent endpoint and the agent key.

If the following environment variables are set before you run the installation, the agent is automatically configured.

export INSTANA_AGENT_KEY=$agentkey
export INSTANA_AGENT_HOST=$endpoint
export INSTANA_AGENT_PORT=$endpoint_port

The endpoint, port, and agent key values are shown on Agent deployment screens in the Instana UI; for example, on a Linux - Automatic Installation (One-liner) screen, the deployment code includes ... -a aGeNTKEY0vaLuO0Eu1ABc ... -e ingress-green-saas.instana.io:443.

If you are using Git-based configuration management, placing this information in a file is a good approach.

For systemd, a drop-in is the easiest approach to overwriting environment specifics for services.

For SysVinit, place the files in either /etc/sysconfig (Red Hat derivatives) or /etc/default (Debian derivatives). For more information, see Setting and Overriding Environment Variables.

Installing the agent

After you refresh the package manager's sources, install one of the following flavors:

Debian Derivatives

To install the static agent, run the following command:

apt-get install instana-agent-static

To install the dynamic agent, run the following command:

apt-get install instana-agent-dynamic

Red Hat Derivatives

To install the static agent, run the following command:

yum install instana-agent-static

To install the dynamic agent, run the following command:

yum install instana-agent-dynamic

Eclipse OpenJ9 11 bundled packages

The package names for the Eclipse OpenJ9 11 bundled agents are instana-agent-static-j9 and instana-agent-dynamic-j9.

Identified problems during installation

You might meet the following problems during installation:

  • Debian-based Derivatives might present a connection error message. This bug is found in GnuTLS-depending programs like Curl, git, or apt:

    GnuTLS recv error (-9): A TLS packet with unexpected length was received
    
  • With Debian-based Derivatives, you might experience the following output when you issue the installation. In this case, install the package apt-transport-https because Apt doesn't currently pull fonts from HTTPS servers.

    E: The method driver /usr/lib/apt/methods/https could not be found.` `APT repository metadata update failed
    
  • In case SUSE Linux Enterprise Server 12 gives you the following output, you need to update your GnuTLS, OpenSSL, and NSS libraries and its dependent Programs (for example Curl). This update is important to properly receive artifacts through modern HTTPS connections:

    Error building the cache:
    [instana-agent|https://_@packages.instana.io/agent/generic/x86_64] Valid metadata not found at specified URL
    Some of the repositories have not been refreshed because of an error.
    No provider of 'instana-agent-static' found.
    Instana agent package install failed
    
  • If you try to enable the instana-agent service by using systemd, you can see the following error message in SUSE Enterprise Linux:

    systemctl enable instana-agent
    
    Synchronizing state of instana-agent.service with SysV service script with /usr/lib/systemd/systemd-sysv-install.
    Executing: /usr/lib/systemd/systemd-sysv-install enable instana-agent
    ln -sf ../instana-agent /etc/init.d/rc2.d/S50instana-agent
    ln: failed to create symbolic link '/etc/init.d/rc2.d/S50instana-agent': No such file or directory
    

    To solve the error, delete the file /etc/init.d/instana-agent, and then run the systemctl enable instana-agent command.

    Then, you can see that the symlink is created correctly:

    systemctl enable instana-agent
    
    Created symlink /etc/systemd/system/multi-user.target.wants/instana-agent.service → /lib/systemd/system/instana-agent.service.`
    
  • On CentOS 6 based systems, outdated libcurl versions can cause this log output:

    Updating YUM metadata ...
    YUM repository metadata update failed
    

To solve the issue, update the libcurl versions.

Checking the status of the host agent

After you install the host agent, you can check the status of the host agent in the Instana UI or on the host. For more information, see Checking the status of the host agent.

Configuring the agent

For more information about the host agent configuration, see Host agent configuration.

Running the agent

To run the Instana agent, use one of the following commands:

  • systemctl start instana-agent.service, if your operating system uses systemd as init system.

  • service instana-agent start, if your operating system uses SysVinit as init system.

Updating the agent

You can use the package manager of your operating system to upgrade an agent package on your machine. To update and manage the packages and patches of your system, run the following command:

yum update // apt-get upgrade

Then, instana-agent-static or instana-agent-dynamic packages are upgraded.

Uninstalling the agent

To uninstall the Instana agent, the first step is to package manager.

Uninstall on Debian Derivatives

Get the name of the package that is installed on your system by running the following command:

apt list --installed | grep instana-agent

The output holds the exact package name to use for the uninstall command. In the following example, the package name is instana-agent-static-j9/generic:

instana-agent-static-j9/generic,now 20221024-1455 amd64 [installed]

Uninstall the agent by running the following command:

sudo apt-get purge <package_name>

Where <package_name> is the package name that you get.

Uninstall on Red Hat Derivatives

Get the name of the package that is installed on your system by running the following command:

yum list installed | grep instana-agent

The output holds the exact package name to use for the uninstall command. In the following example, the package name is instana-agent-static-j9.x86_64:

instana-agent-static-j9.x86_64       20221024-1455                 @instana-agent

Uninstall the agent by running the following command:

sudo yum remove <package_name>

Where <package_name> is the package name that you get.

Note: Some files that are created by the Instana agent are not part of the package on rollout. The process of clean-up the installation of the Instana agent requires an extra manual step to remove the installation directory. The clean-up is required for SUSE and Red Hat derivatives because Yum doesn't support clean-up.

Run the following command to remove the installation directory:

rm -rf /opt/instana/agent

Tarball installation

To install and run the agent from a tarball archive, download and extract the archive, and then start the agent.

The tarball archive is pre-filled with your agent key and host agent endpoint.

The Instana host agent must be installed in a system-wide accessible location and needs to be run as the root user. If you run the host agent with any other user, the functions of the host agent are limited. This is because some of the performance metrics are supported only by the root. The monitoring Docker containers can be done only by the root on the Docker host machine. in addition, only either the root or startup users can attach and monitor Java virtual machines (JVMs). If you can't run the host agent as root, ensure that the agent user is listed in sudoers with a valid shell.

The user that is running the agent needs to be able to write in the agent directory and all its subdirectories. The agent downloads the required sensors in accordance with the auto detection and creates log files in its data subdirectory. Make sure that you have about 100 MB of free disk space.

Prerequisites

A Java Development Kit (JDK) needs to be available for the agent. You can see the list of supported JDKs as follows. You have two options:

  • Place or link the JDK into <instana-agent-install-dir>/jvm (so that <instana-agent-install-dir>/jvm/bin/java exists).

  • The customizable way is to export an environment variable that is called JAVA_HOME to point to that JDK (this environment variable can be also set via instana-agent-install-dir>/bin/setenv).

The following JVMs are supported for running the agent:

The JVM must be executable for all users on the system. You are recommended to use the latest available patch release of the Java distribution of your choice. The agent requires a JDK that supports TLSv1.3 (available in all current JDK 8 and JDK 11 builds). Depending on your OS distribution, the packages that are provided by the OS distributor might not contain strong encryption support due to export control. If you use such a package, you might meet the errors like "java.lang.RuntimeException: Could not generate DH key pair."

To reduce the resources used by the agent when detect file system changes, you need to install the inotify-tools package on your Linux distribution.

Agent communication

The agent downloads updates and sensors from the following host:

DNS Name: artifact-public.instana.io Destination Port: tcp/80 and tcp/443

Currently, the Instana Service is provided in two different regions. Your individual instance is located geographically closest to most of your agents and users. Agents are preconfigured upon download, but some installation methods that require Instana Backend configuration. Consult the Agent Management section inside the product or your technical contact at Instana to learn about the region where your instance is located.

If you are using the self-hosted (on-premises) Instana, use the endpoints that are described at Self-hosted Instana backend on Docker (on-premises).

If you are using Instana SaaS, use the endpoints that are described at host agent endpoints.

Downloading the agent

To download the agent, complete the following steps:

  1. Sign in to the Instana UI, and then select an option to display the agent catalog; for example, on the home page, click Deploy Agent.

    If you are starting a new trial instance of Instana, the agent catalog is displayed with a prompt to select a host agent to install.

  2. Click the tile Linux - Archive (tar.gz)

  3. Select the packaging option to install a dynamic or static agent.

  4. In the dropdown menu, select the platform architecture (operating system).

    The archive file is automatically preconfigured with your Instana account settings so that you only need to extract and start the agent.

  5. Click the download link, and then extract the agent archive.

Download the agent with wget

To download the agent with wget, run the following commands:

wget --save-cookies {{agent_folder_name}}/instana-cookies.txt --post-data 'email={{instana_username}}&password={{instana_password}}' https://{{instana_tenant_unit}}-{{instana_tenant}}.instana.io/auth/signIn
wget --content-disposition --load-cookies {{agent_folder_name}}/instana-cookies.txt --post-data 'type=linux64' https://instana.io/ump/{{instana_tenant}}/{{instana_tenant_unit}}/agent/download -O {{opt_folder}}/{{name_of_agent_archive.tar.gz}}

Starting the agent

To start the agent, run the following command:

INSTANA_AGENT_FOLDER/bin/start

Stopping the agent

To stop the agent, run the following command:

INSTANA_AGENT_FOLDER/bin/stop

Status of the agent

To see the status of the agent, run the following command:

INSTANA_AGENT_FOLDER/bin/status

Running the Instana agent as a non-root user

This feature is experimental. Only the .tar file installation and selected sensors are supported now.

To run the Instana agent as a non-root user, you need to extract the Instana agent as a .tar file.

.tar file installation (RHEL 9)

To install the Instana agent, complete the following steps:

  1. Create a normal user for the Instana agent:

    sudo adduser instana
    
  2. Install Java.

  3. Download the Instana agent .tar file from the agent download page, and then extract the .tar files. For extraction, use GNU tar. After you extract the .tar files, the instana-agent-* directory is displayed.

    tar -xvf <instana-agent-*>.gz
    
  4. Change the owner of the instana-agent directory:

    chown -R instana_user:instana /instana-agent
    
  5. Set environment variables, such as JAVA_HOME, path variables, and other Instana variables:

    export INSTANA_AGENT_KEY=$agentkey
    export INSTANA_AGENT_HOST=$endpoint
    export INSTANA_AGENT_PORT=$endpoint_port
    
  6. Go to the INSTANA_AGENT directory and start the Instana agent:

    #To start agent
    ./bin/start
    
  7. Optional: Check the status of the agent:

    ./bin/status
    

Supported features

The Instana agent that is run as a non-root user supports the following features:

Java attachment

For JVM-based applications, you need to run target JVM application on the same user account where the Instana agent is running so that the attachment works.

Host sensor

The host sensor requires the sudo entry permission in the sudoers file to access readlink command (/proc directory) that is mentioned in the Troubleshooting section.

ACE sensor

If the ACE process is started with the user that starts the Instana agent, everything works well including auto-discovery.

If the ACE process is started with a different user (not the one that starts the Instana agent), then environment variables return empty. In this case, the ACE sensor supports remote monitoring and local monitoring with rest API. But auto-discovery doesn't work.

Troubleshooting a non-root agent

The following issues might be encountered when the Instana agent is started:

  • Issue: Access denied for /var/lib/instana

    java.nio.file.AccessDeniedException: /var/lib/instana
    

    Solution: Create a directory with root privilege and change ownership to an Instana user

    sudo mkdir /var/lib/instana
    sudo chown instana: /var/lib/instana
    sudo chmod 700 /var/lib/instana
    
  • Issue: Memory calculcator error

    Skipping the memory calculator: cannot retrieve the maximum amount of memory to be used by the agent from /sys/fs/cgroup/memory/memory.limit
    The memory calculator has not been used, and neither the 'JAVA_MAX_MEM' nor 'JAVA_OPTS' environment variables specify a maximum heap setting
    

    Solution: This error must not impact the agent itself. The memory calculator is a small binary that is used for systemd or containers to determine how much Java heap to assign to the agent, based on limits set. You can fix it by entering the JAVA_OPTS environment variable.

  • Issue: Password prompting issue that is caused by the agent trying to read the symlink of the process, which is owned by the root.

    Solution:

    • Add following entry in the sudoers file:

      instana ALL=(root)    NOPASSWD:/usr/bin/readlink
      

      This entry allows the agent to access the readlink command for root process.

    • Optional: Debug the issue by enabling sudo logs:

      Defaults logfile=/var/log/sudo.log
      

      The following snippet displays a log where the agent script tries to read the readlink command:

      Nov 12 06:03:19 : instana : command not allowed ; TTY=pts/2 ;
          PWD=/home/tester/instana-agent ; USER=root ; COMMAND=/usr/bin/readlink
          /proc/21025/exe
      Nov 12 06:03:19 : instana : command not allowed ; TTY=pts/2 ;
          PWD=/home/tester/instana-agent ; USER=root ; COMMAND=/usr/bin/readlink
          /proc/21025/cwd
      Nov 12 06:03:19 : instana : command not allowed ; TTY=pts/2 ;
          PWD=/home/tester/instana-agent ; USER=root ; COMMAND=/usr/bin/readlink
          /proc/12152/exe
      Nov 12 06:03:19 : instana : command not allowed ; TTY=pts/2 ;
          PWD=/home/tester/instana-agent ; USER=root ; COMMAND=/usr/bin/readlink
          /proc/12152/cwd
      

Setting and overriding environment variables

When the agent is installed as package and running by SystemV init or systemd, a place is needed to set specific Instana environment variables. Also, it might be necessary to override specific environment variables. For example, for LXC container support, the agent needs to have the LXC command-line utils on its path. See the ways that the environment variables can be overridden.

SystemV Init

The agent sources the corresponding shell scripts in the /etc/default/ directory (Debian and derivatives) or /etc/sysconfig/ directory (Red Hat and derivatives). If the service name is not changed, the SysVinit script sources /etc/default/instana-agent file or /etc/sysconfig/instana-agent file.

For example, in Debian the Instana variables and PATH can be updated by using /etc/default/instana-agent (when this file is found readable):

INSTANA_AGENT_KEY=$agentkey
INSTANA_AGENT_HOST=$endpoint
INSTANA_AGENT_PORT=$endpoint_port
PATH=/usr/local/bin:${PATH}

The endpoint, port, and agent key values are shown on Agent deployment screens in the Instana UI; for example, on a Linux - Automatic Installation (One-liner) screen, the deployment code includes ... -a aGeNTKEY0vaLuO0Eu1ABc ... -e ingress-green-saas.instana.io:443.

Systemd

For systemd, you can find various available options, the drop-in units, EnvironmentFile directives, and global environment settings. Drop-in units are the most recommended approach.

In the following options, make sure to run sudo systemctl daemon-reload followed by sudo systemctl restart instana-agent.service to reload the agent with the variable changes.

Drop-in units

By using drop-in units, you can override the specific settings without changing the original systemd unit file. Drop-ins must be placed in /etc/systemd/system/<unit>.d/ with a name <name>.conf.

For example, to set the Instana and PATH environment variables, create the /etc/systemd/system/instana-agent.service.d/10-environment.conf file with the contents:

[Service]
Environment=INSTANA_AGENT_KEY=$agentkey
Environment=INSTANA_AGENT_HOST=$endpoint
Environment=INSTANA_AGENT_PORT=$endpoint_port
Environment=PATH=/usr/local/bin:${PATH}

EnvironmentFile directives

This solution uses the drop-in units except the environment directives that is read from a separate file. For example, create a /etc/instana/environment.conf file, with the following contents:

INSTANA_AGENT_KEY=$agentkey
INSTANA_AGENT_HOST=$endpoint
INSTANA_AGENT_PORT=$endpoint_port
PATH=/usr/local/bin:${PATH}

Then, in the systemd (drop-in) unit file, use the following configuration:

[Service]
EnvironmentFile=/etc/instana/environment.conf

Global environment settings

By default, systemd reads global configuration from various paths such as /etc/systemd/system.conf or /etc/systemd/system.conf.d/10-default-env.conf. By placing DefaultEnvironment configuration in these files, they become available to all systemd units. For more information, see systemd config.

For example, the following configuration creates three Instana environment variables when you set in the/etc/systemd/system.conf.d/10-default-env.conf file:

[Manager]
DefaultEnvironment="INSTANA_AGENT_KEY=<key>" "INSTANA_AGENT_HOST=<host>" "INSTANA_AGENT_PORT=<port>"

Configuration Files

The following files are created once during the installation:

  • /opt/instana/agent/etc/mvn-settings.xml
  • /opt/instana/agent/etc/org.ops4j.pax.url.mvn.cfg
  • /opt/instana/agent/etc/instana/com.instana.agent.bootstrap.AgentBootstrap.cfg
  • /opt/instana/agent/etc/instana/com.instana.agent.main.config.Agent.cfg
  • /opt/instana/agent/etc/instana/com.instana.agent.main.config.UpdateManager.cfg
  • /opt/instana/agent/etc/instana/com.instana.agent.main.sender.Backend.cfg
  • /opt/instana/agent/etc/instana/configuration.yaml

These files are not overwritten when a user edits them or the package manager upgrades the agent package.

Temporary files

The Instana host agent installs some files that it needs in /tmp/.instana/ directory, if those files are deleted over the uptime of the host, it can cause issues during updates.

Tmpfiles.d

On system.d-based systems, the systemd-tmpfiles deamon can delete files that are needed by the host agent in the /tmp/.instana directory, which causes update failures that are hard to troubleshoot. Since version v181, DEB and RPM agent packages automatically install the following configuration script in /usr/lib/tmpfiles.d that adjusts the behavior to play nice with the Instana host agent:

# Instana files in /tmp cannot be deleted when the agent is running,
# or it may lead to issues on agent update
R! /tmp/.java_pid*
R! /tmp/.instana/*

This configuration allows tmpdfiles.d to delete the files that are needed by the Instana agent only after a host restart. The newly running Instana host agent re-creates the files on startup.

IMPORTANT: Red Hat Enterprise Linux and CentOS, in versions 6 and 7, ship default tmpdfiles.d policies that delete the entire /tmp directory every 10 days regardless of which policies a package like Instana's can specify. It means that a package like instana can define custom policies to control the behavior of the system or the package itself.

#  This file is part of systemd.
#
#  systemd is free software; you can redistribute it and/or modify it
#  under the terms of the GNU Lesser General Public License as published by
#  the Free Software Foundation; either version 2.1 of the License, or
#  (at your option) any later version.

# See tmpfiles.d(5) for details
# Clear tmp directories separately, to make them easier to override
v /tmp 1777 root root 10d
v /var/tmp 1777 root root 30d

...

Short of editing the default tmpfiles.d policies, no known workaround is available. The offending tmpfiles.d policy is located in /usr/lib/tmpfiles.d/tmp.conf.

IMPORTANT: Many Debian derivatives, including Ubuntu, ship default tmpdfiles.d policies that delete the entire /tmp directory regardless of which policies a package like Instana's can specify. Upon installation, the DEB, and RPM packages run a test to verify the behavior of the temporary files tmpfiles.d, running the command systemd-tmpfiles --remove. On systems with policies that are override the wanted behavior that is specified by the Instana agent packages, you might see the following warning:

WARNING: The tmpfiles.d policies of this system may cause the Instana agent to malfunction by deleting the /tmp/.instana directory; refer https://www.instana.com/docs/setup_and_manage/host_agent/on/linux#tmpfiles.d for more information.

Short of editing the default tmpfiles.d policies, no known workaround is available. The offending tmpfiles.d policy is located in /usr/lib/tmpfiles.d/tmp.conf.

Tmpwatch

Red Hat Enterprise Linux 6 and CentOS 6 include the tool tmpwatch, which is known to delete unnecessary files in the /tmp/.instana. However, the use of tmpwatch in these distributions can occasionally cause update problems with the host agent. For more information, see Temporary files.

# Deal with tmpwatch, found in RHEL 6 and CentOS 6.
# It is a cronjob, installed in /etc/cron.d/daily, with pretty much hardcoded
# configurations to wipe data the Instana agent needs.
readonly TMPWATCH_CRONJOB='/etc/cron.d/daily/tmpwatch'
if [ -f "${TMPWATCH_CRONJOB}" ]; then
  # Let's check if we already touched the file and, if not, add our excludes
  if ! grep "${TMPWATCH_CRONJOB}" Instana 2&>1 > /dev/null; then
    echo "tmpwatch discovered in '${TMPWATCH_CRONJOB}': adding the exclude flags for Instana"
    # Edit in place with `sed`
    sed -i '/^flags=/a flags="${flags} -x /tmp/instana -x /tmp/.javapid* # Added by Instana"' "${TMPWATCH_CRONJOB}"
  fi
fi

Troubleshooting agent deployment

If installing the agent is not successful at first, you can check log messages and troubleshooting tips. If this troubleshooting section does not answer the questions you have, contact the IBM Instana support team with information about your experience, so that we can help you and update our documentation accordingly.

For troubleshooting information that is general to all host agents, see Managing host agents / Troubleshooting.


  1. Red Hat Enterprise Linux 6 and CentOS 6 include the tool tmpwatch, which is known to delete necessary files in the /tmp/.instana. However, the use of tmpwatch in these distributions can occasionally cause update problems with the host agent. For more information, see Temporary files. ↩︎

  2. Suse Linux Enterprise Server (SLES) 11 is not supported due to outdated OpenSSL binaries. ↩︎

  3. Instana host agent supports the s390x platform with Red Hat, Suse, and Ubuntu versions that are previously listed and are also supported on IBM Z. ↩︎ ↩︎

  4. Support for Kylin Linux is limited to Mainland China. ↩︎