System requirements for a five-node deployment

Performance isolation

Instana data directories generate high-volume, bursty, and concurrent I/O. Each storage device has a finite I/O capacity that includes:

• IOPS
• Throughput (bandwidth)
• Cache

When multiple directories share a device:

A workload spike in one directory can reduce available capacity for others. It results in unpredictable latency, which can manifest as:

• Query timeouts
• Dropped traces
• Slow dashboards
• Failed health checks

Use dedicated devices to ensure consistent and predictable performance.

Fault containment

Dedicated storage assigns each directory to its own failure domain. If a device becomes full, degraded, or unavailable, only the associated directory is affected.

With shared storage, the same issue impacts all directories sharing that device.

Although Instana data stores are interdependent and full service availability cannot be guaranteed during a failure, isolating storage limits the scope of impact and simplifies recovery operations.

Verifying storage isolation

You must verify that each directory is backed by independent physical resources and that no sharing exists at any layer.

The host can present multiple devices even when they are backed by shared storage. Host-level validation alone is not sufficient.

Validate at the host level, run the following commands to inspect device mappings.

lsblk -o NAME,SIZE,TYPE,MOUNTPOINT

For LVM configurations (trace LV, VG, PV):

sudo pvssudo vgssudo lvdisplay -m

The configuration must meet the following conditions:

• Each mount point uses a distinct block device
• No block device is used by more than one directory

• In LVM setups, each volume group uses only its own physical volumes and no physical volume is shared across volume groups.

Validate at the storage backend:

For SAN, NAS, or virtualised storage, additional validation is required because the host does not have visibility into backend resource sharing.

Confirm the following conditions with your storage provider or administrator:

• The volumes do not originate from the same storage pool, RAID set, or data store.
• Storage controller resources, such as cache, are not shared.
• Fabric or connectivity paths are independent
• I/O activity on one volume does not affect another volume.

Exception: NVMe storage

The isolation requirement is based on characteristics of traditional disks, such as seek latency and serialised I/O queues. NVMe storage significantly reduces these constraints. In NVMe environments, the requirement can be relaxed to a capacity-based model.

Capacity-based placement: Multiple directories can share a single NVMe device if the following conditions are met:

• The device provides sufficient sustained IOPS and throughput.
• Capacity exceeds the combined peak demand of all the directories.

Plan for peak usage, not average usage: The workloads burst, and simultaneous spikes can saturate a device that appears sufficient under steady conditions.

Reduced fault isolation: All directories on the same device share a single failure domain. If the device fails, all colocated directories are affected. Use shared NVMe configurations only if losing that isolation is acceptable.