Instana backend architecture

Architecture diagram

Backend Architecture

Components

Instana backend components are listed in the following table. Some Instana components are only available when the associated feature flags are enabled. For more information, see Enabling optional features.

Component Scope Availability Description
acceptor Shared Always Metrics and traces ingress. Receives raw payloads from the Instana agents and publishes to the unit’s Kafka topic.
accountant Shared Always Collects usage data and contains REST APIs that the Instana UI can use to access the usage data.
appdata-health-aggregator Shared Always Aggregates calls to metric to allow fetching the most recent metric value for a query. It runs as an optimization over querying ClickHouse, as ClickHouse cannot handle many queries. This component is used for Smart Alerts.
action-reader Shared Feature flag feature.automation.enabled Reads action instances from Elasticsearch.
action-writer Shared Feature flag feature.automation.enabled Writes action instances to Elasticsearch.
appdata-health-processor Shared Always Issues recognition for Application perspectives. Processes and manages the state and lifecycle configurations of Smart Alerts, fetches metrics over appdata-health-aggregator, emits event messages to issue-tracker.
appdata-legacy-converter TU Always Allows issue detection by using the existing processing pipeline.
appdata-live-aggregator Shared Feature flag feature.appdata.live.aggregator.enabled Reduces the load against ClickHouse by serving frequent read queries from the cache. High load is caused by the Instana dashboard live view where frequent calls hit appdata-reader and ClickHouse. This component reduces the load.
appdata-processor TU Always Extracts calls from the traces. Receives traces from acceptor, collects the spans for each trace and maps them to infrastructure entities, labels the services, and groups the applications. The information is forwarded to the appdata-writer.
appdata-reader Shared Always Gateway for reading the application data from ClickHouse.
appdata-writer Shared Always Receives processed calls from appdata-processor and eum-processor, and stores them in ClickHouse.
butler Shared Always Public login and user settings management. Supports authentication, authorization, licensing, and accounting.
cashier-ingest Shared Always Pulls incoming usage information from Kafka, calculates usage statistics, and stores it in the PostgreSQL database.
cashier-rollup Shared Always Calculates usage statistics rollups and stores it in the PostgreSQL database.
eum-acceptor Shared Always End-user monitoring (EUM) beacons ingress. Receives beacons and writes them into Kafka topics.
eum-health-processor Shared Always Issues recognition for EUM. Processes EUM data for Smart Alerts of website and mobile apps; emits event messages to issue-tracker.
eum-processor Shared Always EUM processing. Reads beacon from Kafka, set sampling decision, and put them into another Kafka topic.
filler TU Always Reads raw messages from the unit’s Kafka topic that is published by acceptor for building the infrastructure entities and metrics. The infrastructure entities are stored in ElasticSearch. Metrics are stored in ClickHouse and BeeInstana. Entity tags are passed on to tag-processor. The filler component is used for metrics and trace processing and graph maintenance.
gateway Shared Always Central point of contact for EUM and Instana UI.
groundskeeper Shared Always Internal API and access management for Instana agents.
issue-tracker TU Always Reads events from Kafka and time bounds for the issues. Generates and manages events, and sends notifications.
js-stack-trace-translator Shared Always Deminifies JavaScript stacktraces for EUM. Some of the beacons contain stack traces. This component converts minified stack traces to their original source file name and line number by using the soucemap file.
log-processor Shared Feature flag feature.logging.enabled Processes logs from various sources, extracts relevant data, and unifies the data representation to an internal format.
log-reader Shared Feature flag feature.logging.enabled Reads logs and meta information on logs from ClickHouse.
log-writer Shared Feature flag feature.logging.enabled Writes logs from log-processor to ClickHouse. Writes and updates infrastructure tag set.
otlp-acceptor Shared Feature flag feature.otlp.enabled Receives OpenTelemetry data that includes traces, metrics, and logs. The component receives the data in the OpenTelemetry format, transforms them to Instana-native format, and publishes them to the unit’s Kafka topic.
processor TU Always Creates topologies and recognizes issues. Processes infrastructure metrics and all built-in and custom event rules. Emits event messages to issue-tracker.
serverless-acceptor Shared Always Serverless traces ingress. Receives raw payloads from the serverless applications and publishes them to the unit’s Kafka topic.
sli-beacons-filter Shared Feature flag feature.slo.enabled Deserializes the beacons and matches them based on the SLI configurations or Apdex configurations.
sli-calls-filter Shared Feature flag feature.slo.enabled Deserializes the calls and matches them based on the SLI configurations or Apdex configurations.
sli-data-reader Shared Feature flag feature.slo.enabled Retrieves data from ClickHouse to feed the SLO and Apdex widget and their respective APIs.
sli-data-writer Shared Feature flag feature.slo.enabled Writes all the SLI calls and SLI beacons to ClickHouse.
sli-evaluator Shared Always Common logic to calculate SLIs and SLOs based on arbitrary metrics and store them in ElasticSearch.
synthetics-acceptor Shared Feature flag feature.synthetics.enabled Communicates with the Synthetic monitoring PoP (Point of Presence). Registers the PoP by creating a location entry in groundskeeper. Returns Synthetic tests to be run to the PoP upon request. Transfers run results of Synthetic tests to Kafka for the synthetics-writer to store in ClickHouse.
synthetics-health-processor Shared Feature flag feature.synthetic.smart.alerts.enabled The central component for Smart Alerts for Synthetics. Reads the configurations of Smart Alerts for Synthetics from Postgres and the Synthetic test results from Kafka to evaluate Smart Alerts. Emits event messages to issue-tracker when a rule is violated.
synthetics-reader Shared Feature flag feature.synthetics.enabled Interacts with the ui-backend to respond to UI and API requests for Synthetic monitoring information.
synthetics-writer Shared Feature flag feature.synthetics.enabled Stores Synthetic test results into ClickHouse.
tag-processor Shared Always Receives entity tags from filler and stores these tags in ElasticSearch so that the entities can be searched by their tags.
tag-reader Shared Always Allows querying entities by tags in ElasticSearch.
ui-backend TU Always This component is the backend part of the UI and contains the REST APIs that can be used by ui-client.
ui-client Shared Always In-browser part of the UI.

Note: TU-scoped components run one replica per tenant unit.