Monitoring ClickHouse

ClickHouse is is an open source column-oriented database management system. Instana can auto-discover and monitor ClickHouse server processes.

The ClickHouse sensor is automatically deployed and installed after you install the Instana agent.

Sensor (Data Collection)

Tracked Configuration

The Instana agent automatically finds the ClickHouse configuration file (config.xml), either in its default location /etc/clickhouse-server/config.xml or by inspecting the --config-file argument that has been passed when starting the clickhouse-server process. However, the agent currently does not support (that is, does not automatically parse) settings overrides specified in auxiliary configuration files in conf.d/config.d directories next to the main config file (see the Configuration files section of the ClickHouse documentation). ZooKeeper configuration is currently also not automatically taken into account.

The following configuration values are automatically tracked by parsing the ClickHouse configuration file.

  • http_port
  • tcp_port
  • https_port
  • tcp_ssl_port
  • version
  • log
  • error_log
  • interserver_http_port
  • listen_hosts
  • max_connections
  • keep_alive_timeout
  • max_concurrent_queries
  • uncompressed_cache_size
  • mark_cache_size
  • path
  • tmp_path
  • user_config_file_name
  • default_profile
  • default_database
  • timezone
  • umask
  • builtin_dictionaries_reload_interval
  • max_session_timeout
  • default_session_timeout

Metrics

The following metrics are collected:

  • Merge
  • ReplicatedFetch
  • ReplicatedSend
  • ReplicatedChecks
  • BackgroundPoolTask
  • DiskSpaceReservedForMerge
  • DistributedSend
  • QueryPreempted
  • TCPConnection
  • HTTPConnection
  • InterserverConnection
  • OpenFileForRead
  • OpenFileForWrite
  • Read
  • Write
  • SendExternalTables
  • QueryThread
  • ReadonlyReplica
  • LeaderReplica
  • MemoryTracking
  • MemoryTrackingInBackgroundProcessingPool
  • MemoryTrackingForMerges
  • LeaderElection
  • EphemeralNode
  • ZooKeeperWatch
  • DelayedInserts
  • ContextLockWait
  • StorageBufferRows
  • StorageBufferBytes
  • DictCacheRequests
  • Revision
  • RWLockWaitingReaders
  • RWLockWaitingWriters
  • RWLockActiveReaders
  • RWLockActiveWriters

Configuration

The Instana agent automatically finds the ClickHouse config file (config.xml), as described above. The agent can figure out most of the configuration values it needs to connect to ClickHouse by parsing this config file.

The Instana agent accesses ClickHouse via its HTTP interface. In a vanilla ClickHouse installation, this interface is accessible without credentials. In this case, the Instana ClickHouse sensor does not need any special configuration.

If, on the other hand, you have configured the ClickHouse HTTP access to require credentials, you need to provide said credentials to the Instana agent. This can be done by configuring them in the agent's configuration file <agent_install_dir>/etc/instana/configuration.yaml.

Here are all possible configuration values for ClickHouse monitoring:

com.instana.plugin.clickhouse:
  enabled: false
  user: admin
  password: admin
  protocol: http
  host: localhost
  httpPort: 8123
  tracingEnabled: false
  profilingEnabled: false
  clusterName: saas 
  • enabled: The variable enables or disables the sensor for ClickHouse monitoring. Enter true to enable the sensor or false to disable it.
  • user: the username for accessing ClickHouse's HTTP interface,
  • password: the password for accessing ClickHouse's HTTP interface,
  • protocol: either 'http' or 'https',
  • host: the host name or IP of the ClickHouse server
  • httpPort: the port of the ClickHouse server
  • tracingEnabled: The variable enables or disables tracing. Enter true to enable tracing for gathering tracing metrics or false to disable it.
  • profilingEnabled: The variable enables or disables profiling. Enter true to enable profiling for gathering profile data or false to disable it.
  • clusterName: The name of the cluster to monitor.

The configuration value for httpPort needs to be an integer value, otherwise it will be ignored, that is, if it can't be parsed into a number, the standard port will be used.