Troubleshooting the host agent on Linux

See the solutions to the common issues that are observed on the host agent on Linux:

Agents installed by using the automated (one-liner) script

If you are installing the host agent by using the automated (one-liner) script, you might encounter the following problems:

Debian-based

Problem:

With Debian-based derivatives, you might see the following output:

  Setting up Instana APT repository
  Importing Instana GPG key
  Updating apt metadata ...
  E: The method driver /usr/lib/apt/methods/https could not be found.
  APT repository metadata update failed

Solution: In this case, install the package apt-transport-https because Apt doesn't currently pull fonts from HTTPS servers.

SUSE and SLES

Problem: For SUSE Linux Enterprise Server 12, you might see the following output:

  Setting up Instana agent for GNU/Linux
  Setting up Instana zypper repository
  Updating zypper metadata ...
  Installing Instana agent ...
  Error building the cache:
  [instana-agent|https://_@packages.instana.io/agent/generic/x86_64] Valid metadata not found at specified URL
  Some of the repositories have not been refreshed because of an error.
  No provider of 'instana-agent-static' found.
  Instana agent package install failed

Solution: You need to update your openssl to properly receive artifacts through modern HTTPS connections. For network requirements of host agents, see network requirements for Instana agent.

CentOS

Problem: For CentOS-6 based systems, you might see the following output:

Setting up Instana agent for GNU/Linux
Setting up Instana YUM repository
Updating YUM metadata ...
YUM repository metadata update failed

Solution: Update the libcurl versions.

RHEL

For RHEL systems, you might encounter the following problems:

  • Problem: You might see the following output:

    Error unpacking rpm package instana-agent-dynamic-20210630-0948.x86_64
    Error: Transaction failed
    

    Solution: Enabled Federal Information Processing Standard (FIPS) or SELinux can lead to the following error during one-liner installation. Disable FIPS and SELinux when you are installing the host agent.

  • Problem: Manual installation of instana-agent packages fails in the following setup:

    sudo rpm -ivh instana-agent-dynamic-20210713-1352.x86_64.rpm
    ...
    error: unpacking of archive failed on file /etc/init.d/instana-agent;60eda3b3: cpio: Digest mismatch
    error: instana-agent-dynamic-20210713-1352.x86_64: install failed
    

    Solution: A possible workaround for manual installation is to skip verification with the following command:

    sudo rpm -ivh --nodigest --nofiledigest instana-agent-dynamic-20210713-1352.x86_64.rpm
    

Agents installed by using a package manager or manually (DEB, RPM)

If you are installing the host agent by using a package manager or manually (DEB, RPM), you might meet the following problems:

Debian-based

For Debian-based Derivatives, you might meet the following problems:

  • Problem: You might see the following connection error message:

    GnuTLS recv error (-9): A TLS packet with unexpected length was received
    

    Solution: This bug is found in GnuTLS-depending programs like Curl, Git, or apt.

  • Problem: You might see the following output when you issue the installation.

    E: The method driver /usr/lib/apt/methods/https could not be found.` `APT repository metadata update failed
    

    Solution: In this case, install the package apt-transport-https because Apt doesn't currently pull fonts from HTTPS servers.

SUSE and SLES

For SUSE Linux Enterprise Server 12, you might encounter the following problems:

  • Problem: You might see the following output:

    Error building the cache:
    [instana-agent|https://_@packages.instana.io/agent/generic/x86_64] Valid metadata not found at specified URL
    Some of the repositories have not been refreshed because of an error.
    No provider of 'instana-agent-static' found.
    Instana agent package install failed
    

    Solution: You need to update your GnuTLS, OpenSSL, and NSS libraries and its dependent Programs (for example Curl). This update is important to properly receive artifacts through modern HTTPS connections.

  • Problem: If you try to enable the instana-agent service by using systemd, you might see the following error message in SUSE Enterprise Linux:

    systemctl enable instana-agent
    
    Synchronizing state of instana-agent.service with SysV service script with /usr/lib/systemd/systemd-sysv-install.
    Executing: /usr/lib/systemd/systemd-sysv-install enable instana-agent
    ln -sf ../instana-agent /etc/init.d/rc2.d/S50instana-agent
    ln: failed to create symbolic link '/etc/init.d/rc2.d/S50instana-agent': No such file or directory
    

    Solution:

    1. Delete the /etc/init.d/instana-agentfile.

    2. Run the following command.

      systemctl enable instana-agent
      

      Then, you can see that the symlink is created correctly:

      systemctl enable instana-agent
      
      Created symlink /etc/systemd/system/multi-user.target.wants/instana-agent.service → /lib/systemd/system/instana-agent.service.`
      

CentOS

On CentOS 6-based systems, you might encounter the following problem:

Problem: You might see the following log output:

Updating YUM metadata ...
YUM repository metadata update failed

Solution: Update the libcurl versions.

Agents installed and run as a non-root user

When you start the Instana agent as a non-root user, you might see the following problems:

  • Problem: You might see the following output:

    java.nio.file.AccessDeniedException: /var/lib/instana
    

    Solution: Create a directory with the root privilege and change ownership to an Instana user.

    sudo mkdir /var/lib/instana
    sudo chown instana: /var/lib/instana
    sudo chmod 700 /var/lib/instana
    
  • Problem: You might see the following output:

    Skipping the memory calculator: cannot retrieve the maximum amount of memory to be used by the agent from /sys/fs/cgroup/memory/memory.limit
    The memory calculator has not been used, and neither the 'JAVA_MAX_MEM' nor 'JAVA_OPTS' environment variables specify a maximum heap setting
    

    Solution: This error must not impact the agent itself. The memory calculator is a small binary that is used for systemd or containers to determine how much Java heap to assign to the agent, based on limits set. You can fix it by entering the JAVA_OPTS environment variable.

  • Problem: You might see the password prompting issue, which is caused by the agent trying to read the symlink of the process owned by root.

    Solution:

    1. Add the following entry in the sudoers file:

      instana ALL=(root)    NOPASSWD:/usr/bin/readlink
      

      This entry allows the agent to access the readlink command for the root process.

    2. Optional: Debug the issue by enabling sudo logs:

      Defaults logfile=/var/log/sudo.log
      

      The following snippet displays a log where the agent script tries to read the readlink command:

      Nov 12 06:03:19 : instana : command not allowed ; TTY=pts/2 ;
          PWD=/home/tester/instana-agent ; USER=root ; COMMAND=/usr/bin/readlink
          /proc/21025/exe
      Nov 12 06:03:19 : instana : command not allowed ; TTY=pts/2 ;
          PWD=/home/tester/instana-agent ; USER=root ; COMMAND=/usr/bin/readlink
          /proc/21025/cwd
      Nov 12 06:03:19 : instana : command not allowed ; TTY=pts/2 ;
          PWD=/home/tester/instana-agent ; USER=root ; COMMAND=/usr/bin/readlink
          /proc/12152/exe
      Nov 12 06:03:19 : instana : command not allowed ; TTY=pts/2 ;
          PWD=/home/tester/instana-agent ; USER=root ; COMMAND=/usr/bin/readlink
          /proc/12152/cwd
      

If this topic does not help you to resolve the agent installation or starting issue, contact the IBM Instana support team.

For troubleshooting information that is general to all host agents, see Managing host agents: Troubleshooting.