Administering the Instana agent on Kubernetes

After you install the agent, configure and manage the agent as you require. See the following list for all the available agent configuration options on Kubernetes.

Checking the status of the host agent

After you install the host agent, check the status of the host agent on the Instana UI or on the host. For more information, see Checking the status of the host agent.

Configuring the agent

You can configure the Instana agent for Kubernetes and Red Hat OpenShift in two different ways, depending on the method that you used to install the agent.

Configuring agent installed by using operator

To configure the Instana agent that is installed by using an operator, complete the following steps:

  1. Open the custom resource YAML file instana-agent.customresource.yaml.

  2. Save your configurations under spec.agent.configuration_yaml. See the following example:

    spec:
      agent:
        configuration_yaml: |
          # Example of configuration yaml template
          # Host
          com.instana.plugin.host:
            tags:
              - 'dev'
              - 'app1'
    
  3. Apply the changes by running the following command:

    kubectl apply -f instana-agent.customresource.yaml
    

For a detailed example of the YAML file, see instana_v1_extended_instanaagent.yaml.

For more information about agent configuration, see Configuring host agents by using the agent configuration file.

Configuring agent installed by using Helm chart

To configure the Instana agent that is installed by using Helm chart, choose either of the following methods:

  • Specify the configuration on the command line by using the flag --set agent.configuration_yaml=<config>.

  • Add the configuration to the Helm chart configuration file values.yaml. Pass the file to the Helm install (during installation) or Helm upgrade (after installation) command by using the -f flag.

For a detailed list of all the configuration parameters, see Instana Helm Chart.

Customizing the agent configuration

After you install the host agent, the following configuration files are created:

/opt/instana/agent/etc/mvn-settings.xml
/opt/instana/agent/etc/org.ops4j.pax.url.mvn.cfg
/opt/instana/agent/etc/instana/com.instana.agent.bootstrap.AgentBootstrap.cfg
/opt/instana/agent/etc/instana/com.instana.agent.main.config.Agent.cfg
/opt/instana/agent/etc/instana/com.instana.agent.main.config.UpdateManager.cfg
/opt/instana/agent/etc/instana/com.instana.agent.main.sender.Backend.cfg
/opt/instana/agent/etc/instana/configuration.yaml

You can customize the configuration of the agent as you need. For example, you can change the TCP port and hostname of the Instana backend and services that your host agent connects to, configure the agent to report to multiple backends, or configure use of a proxy for indirect communication from the host agent to the Instana backend.

For more information about customizing the agent configuration, see the following topics:

Starting and stopping the agent

Starting the agent

When you install the host agent, it is enabled to start automatically at system boot. You can turn off this behavior by setting the environment variable INSTANA_AGENT_AUTOSTART=false when you install the agent.

  • To start the host agent, run the following command:

    INSTANA_AGENT_FOLDER/bin/start
    
  • If your operating system uses systemd as init system, run the following command:

    systemctl start instana-agent.service
    
  • If your operating system uses SysVinit as init system, run the following command:

    service instana-agent start
    

Stopping the agent

To stop the agent, run the following command:

INSTANA_AGENT_FOLDER/bin/stop

Updating the agent installation

Depending on your method of installation, you can update the Instana agent for Kubernetes in multiple ways.

Updating an operator installation

Updating the agent installation involves two steps:

  1. Updating the operator
  2. Updating the images

To update the operator, complete the following steps:

  1. On the home page of the Instana UI, click Deploy Agent.

  2. On the agent deployment catalog page, click the tile OpenShift - Operator.

  3. Copy and run the command to deploy the new operator. See the following command sample:

    kubectl apply -f https://github.com/instana/instana-agent-operator/releases/latest/download/instana-agent-operator.yaml
    

After the operator is updated, proceed to updating the images.

Updating a Helm chart installation

Updating the agent installation involves two steps:

  1. Updating the Helm chart
  2. Updating the images

To update the agent Helm chart, run the following command:

helm pull --repo https://agents.instana.io/helm --untar instana-agent && kubectl apply -f instana-agent/crds; helm upgrade --namespace instana-agent instana-agent \
--repo https://agents.instana.io/helm instana-agent \
--reuse-values

Instana agent Helm charts version 2.0.0 and earlier do not have the instana-agent/crds folder. Therefore, the kubectl apply -f instana-agent/crds step is not required.

After the Helm chart is updated, proceed to updating the images.

Updating the images

The instana-agent DaemonSet and k8sensor deployment are set to pull the latest image on pod startup. To force the update, run the following commands:

kubectl rollout restart ds instana-agent -n instana-agent
kubectl rollout restart deployment k8sensor -n instana-agent

Managing agents by using the Agent Management dashboard

You can complete the following operations by using the Agent Management dashboard in the Instana UI:

To open the Agent Management dashboard, complete the following steps:

  1. In the sidebar of the Instana UI, select Infrastructure.
  2. Click a specific monitored host.
  3. On the host dashboard, click Open Agent Management.

Configuring network access for monitored applications

You must configure the network access of the following types of applications for them to communicate with the agent:

  • Node.js
  • Go
  • Ruby
  • Python
  • .NET Core

These applications need to know the IP that the agent listens. The host agent listens on the host IP automatically. You can use the following Downward API snippet to pass the host IP in an environment variable to the application pod:

spec:
  containers:
    env:
      - name: INSTANA_AGENT_HOST
        valueFrom:
          fieldRef:
            fieldPath: status.hostIP

Monitoring master nodes

By default, the host agent does not get scheduled on Kubernetes master nodes, because the deployment respects the default taint node-role.kubernetes.io/master:NoSchedule that is set on most master nodes. To monitor master nodes, add the following toleration to the agent Daemonset:

kind: DaemonSet
metadata:
  name: instana-agent
  namespace: instana-agent
spec:
  template:
  ...
    spec:
      tolerations:
        - key: "node-role.kubernetes.io/master"
          effect: "NoSchedule"
          operator: "Exists"
    ...

For more direct control, install the host agent separately on the master nodes.

Monitoring Kubernetes NGINX Ingress

To configure Kubernetes NGINX Ingress and the host agent for capturing NGINX metrics, see Monitoring NGINX.

To enable distributed tracing for Kubernetes NGINX Ingress, see Distributed Tracing for NGINX Ingress.

Configuring agent secrets

Kubernetes has built-in support for storing and managing sensitive information. However, if you do not use this built-in capability but still need the ability to redact sensitive data in Kubernetes resources, you can configure the agent secrets.

To enable sensitive data redaction for selected Kubernetes resources (specifically annotations and container environment variables), complete the following steps:

  1. Set the INSTANA_KUBERNETES_REDACT_SECRETS environment variable to true as in the following agent YAML snippet:

    spec:
      containers:
          env:
            - name: INSTANA_KUBERNETES_REDACT_SECRETS
              value: "true"
    
  2. Configure the host agent with the desired list of secrets as described in the agent secrets configuration.

You might notice a decrease in performance of the Kubernetes sensor on enabling this capability.

Reporting to multiple backends

For the host agent that is installed by using the Operator or the Helm chart, you can add additional backends by running the following command:

When you run the following command, only the agent data is forwarded to multiple backends. The Kubernetes sensor and metrics data is not forwarded to multiple backends.

  • For the host agent that is installed by using Helm chart:

       helm install instana-agent \
       --repo https://agents.instana.io/helm \
       --namespace instana-agent \
       --create-namespace \
       --set agent.key=my-key \
       --set agent.endpointHost='<your_host_agent_endpoint>' \
       --set agent.endpointPort=443 \
       --set cluster.name='<your_cluster_name>' \
       --set zone.name='zone-name' \
       --set "agent.additionalBackends[0].endpointHost=<your_host_agent_endpoint>" \
       --set "agent.additionalBackends[0].key=<your_agent_key>" \
       --set "agent.additionalBackends[0].endpointPort=443" \
       --set "agent.additionalBackends[1].endpointHost=<your_other_host_agent_endpoint>" \
       --set "agent.additionalBackends[1].key=<your_other_agent_key>" \
       --set "agent.additionalBackends[1].endpointPort=443" \
       instana-agent
    
  • For the host agent that is installed by using Operator:

    apiVersion: instana.io/v1
    kind: InstanaAgent
    metadata:
      name: instana-agent
      namespace: instana-agent
    spec:
      zone:
        name: edited-zone # (optional) name of the zone of the host
      cluster:
        name: my-cluster
      agent:
        endpointHost: <your_host_agent_endpoint>
        endpointPort: "443"
        env: {}
        key: <your_agent_key>
        additionalBackends:
        - endpointHost: <your_host_agent_endpoint>
          endpointPort: "443"
          key: <your_agent_key>
        configuration_yaml: |
          # You can leave this empty, or use this to configure your instana agent.
          # See https://github.com/instana/instana-agent-operator/blob/main/config/samples/instana_v1_extended_instanaagent.yaml for the extended version.
    

For more information about reporting to multiple backends from a Kubernetes agent, see Configuring Instana backend.