System requirements for a multi-node deployment

Edit online

Review the system requirements of Self-Hosted Standard Edition on a multi-node cluster.

The specifications in this section represent minimum requirements for a functional deployment.

Each self hosted environment has upper limits for the load it can process. Your actual resource needs may vary based on factors such as:
  • The number of servers under observation.
  • The technologies that run on the observed servers, as some technologies are more metric intensive.
  • The load on an observed environment, from a request perspective, is determined by the trace load, which is directly proportional to both the incoming traffic to the observed system and its internal traffic.
  • Application architecture and traffic patterns, high-traffic microservices generate far more spans than monolithic applications.
  • Infrastructure complexity, a Kubernetes node with hundreds of pods can produce significantly more metrics than a single VM.
  • Enabled features, like End-User Monitoring (EUM), logging, and serverless monitoring can increase data ingestion and storage.
  • Traffic spikes and seasonal events, take into consideration peak periods like Black Friday or promotional campaigns.
Note: Contact your IBM representatives to identify whether your environment fits into the selected deployment approach. For larger workloads, you can increase resources beyond the minimum requirements to ensure optimal performance.

Hosts

Edit online

For multi-node environments, you need three hosts with the same operating system.

The three nodes in a multi-node cluster have specific uses:

  • The first node (instana-0), which is labeled as node-role.instana.io: "backend" during installation, is used for running backend workloads that require persistent volumes. The node also runs the gateway, acceptors, and the UI backend.

  • The second node (instana-1), which is labeled as node-role.instana.io: "datastore" during installation, is used for running data stores.

  • The third node (instana-2), which is labeled as node-role.instana.io: "other" during installation, is used for running the rest of Instana workloads.

Supported platforms and operating systems

Edit online

Make sure you have a host on which you can run the Self-Hosted Standard Edition installer.

Note: Important: The host can be a virtual machine or a dedicated physical machine. The host must be new with a freshly installed operating system. If you want to use a host that was previously used for something else, you must reinstall the operating system. keyword.self_se must not co-exist with a keyword.self_docker.

Instana supports the following platforms and operating systems for the host:

Table 1. Supported operating systems
Platform Operating system
Linux® x86_64 Red Hat® Enterprise Linux® (RHEL) 9 and 8
Ubuntu 24.04 and 22.04
Debian 12 and 11
CentOS Stream 9
Amazon Linux 2023
Oracle Linux 9
SUSE Linux Enterprise Server (SLES) 15 SP6 SP7
Linux® arm64

Supported on AWS Graviton

Deployment on other operating systems on arm64 is not tested.

Hardware requirements

Edit online

In multi-node clusters, only the production installation type is supported.

CPU and memory requirements

Edit online

Each node may require, up to the specified CPU and memory in the following table.

Table 2. CPU and memory requirements
Number of CPU cores Memory (GB)
12 48

For each optional feature that you want to use, you need additional resources. The following table outlines the CPU and memory requirements of the optional features for the production installation type.

Table 3. CPU and memory requirements for optional features
Optional feature Number of CPU cores Memory (GB)
Action framework 1.5 7
Analyze logs 2 11
Business monitoring 1 5
end user monitoring 2 4.5
Self monitoring 1 2
Synthetic monitoring 2 8

Provision the additional resources, such as CPU and memory capacity on node0 (instana0) and node2 (instana-2).

Note: Do not provision the resources on node1 (instana-1) as node1 is primarily used for the data stores. In addition, if you enable the logging feature, then you need more disk space to store the log data on node1 (instana-1).

Storage requirements

Edit online

Each node may require, up to the specified storage in the following table.

The disk storage speed requirements measured as input/output operations per second (IOPS).

Table 4. Disk IOPS requirements
Node IOPS
VM1 and VM2 (deployments) 3000 (data transfer at 250 MiB/second )
VM3 (Stateful sets)

3000 (data transfer at 250 MiB/second) without the Analyze Logs feature

6000 (data transfer at 500 MiB/second) with the “Analyze Logs” feature

The four data storage directories are required on the following nodes. You must add additional disks on these nodes.

  • Objects directory on node0 (instana-0).
  • Data, metrics, and analytics directories on node1 (instana-1).

No additional disk is needed on node2 (instana-2).

The following table provides the storage volume requirements of each node:

Table 5. Cluster storage volume requirements for node0 instana-0
Description Default directory Min size (GB) Comments
Object directory /mnt/instana/stanctl/objects 1000 Override path with --volume-objects
Root directory / 100
$HOME directory (air-gapped) /root or the home directory of users 60 Override with $HOME environment variable. 40 for $HOME directory (air-gapped) and 20 GB for separate disk space.
$HOME directory (non-air-gapped) /root or the home directory of users 10 Override with $HOME environment variable
Cluster data directory /var/lib 100 Override with --cluster-data-dir
Table 6. Cluster storage volume requirements for node1 instana-1
Description Default directory Min size (GB) Comments
Data directory /mnt/instana/stanctl/data 500 Override path with --volume-data
Metrics directory /mnt/instana/stanctl/metrics 1000
Analytics directory /mnt/instana/stanctl/analytics 1200
Root directory / 100
$HOME directory (air-gapped) /root or the home directory of users 60 Override with $HOME environment variable. 40 for $HOME directory (air-gapped) and 20 GB for separate disk space.
$HOME directory (non-air-gapped) /root or the home directory of users 10 Override with $HOME environment
Cluster data directory /var/lib 100 Override with --cluster-data-dir
Table 7. Cluster storage volume requirements for node2 instana-2
Description Default directory Min size (GB) Comments
Root directory / 100
$HOME directory (air-gapped) /root or the home directory of users 60 Override with $HOME environment variable. 40 for $HOME directory (air-gapped) and 20 GB for separate disk space.
$HOME directory (non-air-gapped) /root or the home directory of users 10 Override with $HOME environment
Cluster data directory /var/lib 100 Override with -—cluster-data-dir
Note: Note: In an air-gapped environment, you need 20 GB more disk space in the directory where you create the air-gapped package, both on the bastion host and the Instana host. For more information about the package, see Creating the air-gapped installation package.

For about a month, you must monitor the storage volume that you initially assigned. You can then increase the capacity if necessary. Also, if you use more agents or enable more optional features, you must monitor the memory usage.

Default directories

Edit online

Instana uses the following directories for data storage.

  • Data directory: Used for Elasticsearch, PostgreSQL, and Kafka data stores. The default location is /mnt/instana/stanctl/data.
  • Metrics directory: Used for Cassandra and BeeInstana data stores. The default location is /mnt/instana/stanctl/metrics.
  • Analytics directory: Used for ClickHouse data store. The default location is /mnt/instana/stanctl/analytics.
  • Objects directory: Used for raw spans, Synthetic monitoring, and end user monitoring (EUM). The default location is /mnt/instana/stanctl/objects.
  • Cluster data directory: Used for cluster data. The default location is /var.
  • $HOME directory: Home directory of the current root or non-root user.

Due to the high data throughput of various Instana features and to avoid performance issues, use fast, dedicated storage such as solid-state drives (SSDs). Use separate disks for each of the data, metrics, analytics, and objects directories.

If you are using separate disks, you must mount the following directories onto the disks:

  • Cluster data directory: You can specify a custom directory if you don’t have enough disk space in /var. For example, /xyz/data. Make sure to specify this directory by using the --cluster-data-dir during installation.
  • $HOME directory: $HOME refers to the home directory of the current user. For example, if the user is a root user, $HOME would be /root; for a non-root user, $HOME would be /home/<username>.

Networking requirements

Edit online

Make sure that you meet the following ports and IP addresses requirements. For more information about opening these ports, see Firewall rules.

Note: Make sure that you have a high-speed, low-latency, and stable connection on the backend server to pull the necessary packages.

Required ports

Edit online

The following ports on all nodes must be open and accessible.

Table 8. Multi-node ports
Port number Direction Protocol Source Description
22 Inbound TCP External Port required for Secure Shell (SSH) connection (required only if you want to log in SSH)
22 Inbound TCP Internal SSH port required for access between backend nodes
80 Inbound TCP External HTTP protocol for Instana console UI
443 Inbound TCP External HTTPS protocol for Instana console UI, Instana console API, Instana EUM, OpenTelemetry, and Instana agent acceptor port
443 Outbound TCP External Required only in online environments. For more information, see Outbound network access requirements for self-hosted Instana deployments.
53 Inbound TCP/UDP Internal (all nodes) DNS port required for resolving domain names
6443 Inbound TCP Internal (all nodes) Internal service port
8443 Inbound TCP External Instana agent acceptor port. This port is optional.
10250 Inbound TCP Internal (all nodes) Internal service port
2379 Inbound TCP Internal (all nodes) Internal service port
2380 Inbound TCP Internal (all nodes) Internal service port
5001 Inbound TCP Internal (all nodes) Internal service port
8472 Inbound UDP Internal (all nodes) Internal service port
9443 Inbound TCP Internal (all nodes) Internal service port
all Inbound TCP/UDP 10.42.0.0/16 and 10.43.0.0/16 Subnets of the self-hosted Instana components
all Inbound TCP/UDP Loopback Open the ports to allow a VM to send and receive its own data packets.

See the following notes:

  • External source means that the port must be accessible from outside of the Instana self-hosted enterprise (private) network.
  • Internal source means that the port on one node must be accessible by the other two nodes of the multi-node cluster.
  • IP addresses 10.42.0.0/16 and 10.43.0.0/16 must be able to access all ports (1 - 65535) internally.
  • The firewall must trust all traffic from the loopback address.
  • Port 443 outbound is needed for accessing certain repositories. For more information, see Outbound network access requirements.

IP addresses

Edit online

The three nodes of your multi-node cluster must meet the following IP address requirements:

  • All the three nodes need a private IP address on the same private VLAN to be able to communicate with each other.
  • node0 (instana-0) must also have a IP address for external communication. The IP address is also used to reach the hosts from outside the cluster.