Monitoring statsd

Sensor (Data Collection)

The Instana agent acts as a statsd collector daemon, and can receive metrics in the same way as a real statsd daemon. So you don't need to start another statsd daemon. If the standard statsd protocol of any client library is used, when data is sent to the sensor, up to 3000 custom metrics are displayed on the host dashboard. The host is where the agent receives the metric data. At most 1000 metrics can be collected for each metric type: counters, gauges, timers.

To view the metrics, select Infrastructure in the sidebar of the Instana User interface, click a specific monitored host, and then you can see a host dashboard with all the collected metrics and monitored processes.

Configuration

By default, the statsd sensor is disabled. To enable the sensor, edit the agent configuration file <agent_install_dir>/etc/instana/configuration.yaml:

com.instana.plugin.statsd:
  enabled: true
  ports:
    udp: 8125
    mgmt: 8126
  bind-ip: "0.0.0.0" # all IPs by default
  flush-interval: 10 # in seconds

Note: Do not run the regular statsd service, it will prevent the agent from listening on these ports. Run only the agent, which will act as statsd server.

Metrics transmission

Via UDP, metrics are sent to the configured UDP port in the statsd format. For information about the format, see the statsd documentation.

For example, the following script increments a counter called hits by one:

echo "hits:1|c" | nc -u -w0 127.0.0.1 8125

Using \n as a separator, multiple metrics are sent via UDP to the configured UDP port in the statsd format:

echo "hits:1|c\nlatency:320|ms\nvisits:333|g\nresponse time:765|s" | nc -u -w0 127.0.0.1 8125

The sensor also accepts metrics format that includes tags:

echo "host.network_receive_errs_total:10|c|#collector:network,device:utun2,host:testmachine" | nc -u -w0 127.0.0.1 8125

Note: To check whether the UDP port is open to Instana StatsD sensor, follow the steps:

  1. Disable StatsD sensor.
  2. Run the command nc -l -u 8125 on the machine where the request will be accepted (the machine where the Instana agent is installed with activated StatsD sensor).
  3. Send a command from the other machine, such as echo -n "custom.metric.name-test:1|c" | 34.135.37.172 8125.
  4. If the metric is showed in the output of the listener that is running, on the port 8125, then the connection to StatsD is established. If not, then UDP port is not open on that machine.