Installing the Instana Backend

Secret instana-tls

The instana-tls secret is required for ingress configuration, and must be created in the core namespace.

Note: This secret must be of type kubernetes.io/tls.

See the following example:

# Generate self signed certificate
openssl req -x509 -newkey rsa:2048 -keyout tls.key -out tls.crt -days 365 -nodes -subj "/CN=<FQDN-Of-Your-Instana>"

# Create a secret holding newly generated certificate
kubectl create secret tls instana-tls --namespace instana-core \
    --cert=path/to/tls.cert \
    --key=path/to/tls.key

Core Secret

Create a config.yaml file with the contents as shown in the following example. The structure of the file is exactly the same as that of the CoreSpec that will be created further as follows.

Secret values must go into this secret. Anything else goes into the CoreSpec.

# Diffie-Hellman parameters to use
dhParams: |
  -----BEGIN DH PARAMETERS-----
  <snip/>
  -----END DH PARAMETERS-----
# The download key you received from us
downloadKey: mydownloadkey
# The sales key you received from us
salesKey: mysaleskey
# Seed for creating crypto tokens. Pick a random 12 char string
tokenSecret: mytokensecret
# Configuration for raw spans storage
storageConfigs:
  rawSpans:
    # Required if using S3 or compatible storage bucket.
    # Credentials should be configured.
    # Not required if IRSA on EKS is used.
    s3Config:
      accessKeyId: ...
      secretAccessKey: ...
    # Required if using Google Cloud Storage.
    # Credentials should be configured.
    # Not required if GKE with workload identity is used.
    gcloudConfig:
      serviceAccountKey: ...

# SAML/OIDC configuration
serviceProviderConfig:
  # Password for the key/cert file
  keyPassword: mykeypass
  # The combined key/cert file
  pem: |
    -----BEGIN RSA PRIVATE KEY-----
    <snip/>
    -----END RSA PRIVATE KEY-----
    -----BEGIN CERTIFICATE-----
    <snip/>
    -----END CERTIFICATE-----
# Required if a proxy is configured that needs authentication
proxyConfig:
  # Proxy user
  user: myproxyuser
  # Proxy password
  password: my proxypassword
emailConfig:
  # Required if SMTP is used for sending e-mails and authentication is required
  smtpConfig:
    user: mysmtpuser
    password: mysmtppassword
  # Required if using for sending e-mail.
  # Credentials should be configured.
  # Not required if using IRSA on EKS.
  sesConfig:
    accessKeyId: ...
    secretAccessKey: ...
# Optional custom CA certificate to be added to component trust stores
# in case internal systems Instana talks to (e.g. LDAP or alert receivers) use a custom CA.
customCACert: |
  -----BEGIN CERTIFICATE-----
  <snip/>
  -----END CERTIFICATE-----
  # Add more certificates if you need
  # -----BEGIN CERTIFICATE-----
  # <snip/>
  # -----END CERTIFICATE-----

• Diffie-Hellman parameters can be generated by using the following command:

An encrypted key for signing/validating messages that are exchanged with the IDP must be configured. Unencrypted keys won't be accepted.

The following commands can be used to create a combined key/cert file:

# Create the key
openssl genrsa -aes128 -out key.pem 2048

# Create the certificate
openssl req -new -x509 -key key.pem -out cert.pem -days 365

# Combine the two into a single file
cat key.pem cert.pem > sp.pem

Assuming the Core object's name is instana-core, create a secret with the same name in the Core's namespace.

kubectl create secret generic instana-core --namespace instana-core --from-file=path/to/config.yaml

Unit Secret

Create a config.yaml file with the contents as shown in the following example. The structure of the file is exactly the same as that of the UnitSpec that will be created further as follows.

Secret values must go into this secret. Anything else goes into the UnitSpec.

# The initial user of this tenant unit with admin role, default admin@instana.local.
# Must be a valid e-maiol address.
# NOTE:
# This only applies when setting up the tenant unit.
# Changes to this value won't have any effect.
initialAdminUser: myuser@example.com
# The initial admin password.
# NOTE:
# This is only used for the initial tenant unit setup.
# Changes to this value won't have any effect.
initialAdminPassword: mypass
# The Instana license. Can be a plain text string or a JSON array encoded as string. Deprecated. Use 'licenses' instead. Will no longer be supported in release 243.
license: mylicensestring # This would also work: '["mylicensestring"]'
# A list of Instana licenses. Multiple licenses may be specified.
licenses: [ "license1", "license2" ]
# A list of agent keys. Specifying multiple agent keys enables gradually rotating agent keys.
agentKeys:
  - myagentkey

Assuming the Unit object's name is tenant0-unit0, create a secret with the same name in the Unit's namespace.

kubectl create secret generic tenant0-unit0 --namespace instana-units --from-file=path/to/config.yaml

S3 (or compatible)

spec:
  storageConfigs:
    rawSpans:
      s3Config:
        # Endpoint address of the object storage.
        # Doesn't usually need to be set for S3.
        endpoint:
        # Region.
        region: eu-central-1
        # Bucket name.
        bucket: mybucket
        # Prefix for the storage bucket.
        prefix: myspans
        # Storage class.
        storageClass: Standard
        # Bucket name for long-term storage.
        bucketLongTerm: mybucket
        # Prefix for the long-term storage bucket.
        prefixLongTerm: myspans-longterm
        # Storage class for objects that are written to the long-term bucket.
        storageClassLongTerm: Standard

  # If using IRSA
  serviceAccountAnnotations:
    eks.amazonaws.com/role-arn: arn:aws:iam::111122223333:role/my-role

accessKeyId and secretAccessKey must go into the Core secret as specified in the section 4.1.3.2. Alternatively, IAM roles for service accounts are supported.

Google Cloud Storage (GCS)

spec:
  storageConfigs:
    rawSpans:
      gcloudConfig:
        # Endpoint address. Doesn't usually need to be set for S3.
        endpoint:
        # Region.
        region: eu-central-1
        # Bucket name.
        bucket: mybucket
        # Prefix for the storage bucket.
        prefix: myspans
        # Storage class.
        storageClass: Standard
        # Bucket name for long-term storage.
        bucketLongTerm: mybucket
        # Prefix for the long-term storage bucket.
        prefixLongTerm: myspans-longterm
        # Storage class for objects that are written to the long-term bucket.
        storageClassLongTerm: Standard

  # If using Workload Identity
  serviceAccountAnnotations:
    iam.gke.io/gcp-service-account: rawspans@myproject.iam.gserviceaccount.com

The serviceAccountKey must go into the Core secret as specified in the section 4.1.3.2. Alternatively, Workload Identity is supported.

Filesystem

You must configure pvcConfig, which is a PersistentVolumeclaimSpec. The volume must have access mode ReadWriteMany.

spec:
  storageConfigs:
    rawSpans:
      pvcConfig:
        accessModes:
          - ReadWriteMany
        resources:
          requests:
            storage: 100Gi
        volumeName: my-nfs-volume
        storageClassName: ""

Infra metrics retention settings

The following retention properties are for the metric rollup tables, and the listed values are the defaults in seconds. A value of zero tells the system to not drop rollups of this time span. A zero value for smaller rollups can cause the disks to quickly fill up.

kind: Core
metadata:
  name: instana-core
  namespace: instana-core
spec:
...
  properties:
    - name: retention.metrics.rollup5
      value: "86400"
    - name: retention.metrics.rollup60
      value: "2678400"
    - name: retention.metrics.rollup300
      value: "8035200"
    - name: retention.metrics.rollup3600
      value: "34214400"
...

Application retention settings

Application Perspectives and End User Monitoring share the short-term data retention property, with the default of 7 days. Beyond the retention period of short-term data, the long-term data is still kept for 13 months and is not configurable.

Note: Only values greater than or equal to 7 days are valid. Values that are less than 7 days lead to a failure of the entire system.

kind: Core
metadata:
  name: instana-core
  namespace: instana-core
spec:
...
  properties:
    - name: config.appdata.shortterm.retention.days
      value: "7"
...

See the following impact on Application Perspectives data:

Examples:

• config.appdata.shortterm.retention.days is changed from 7 days to 14 days:
  Old data is deleted once it is older than 7 days, and new data is deleted once it is older than 14 days.
• config.appdata.shortterm.retention.days is changed from 14 days to 7 days:
  Old data is deleted once it is older than 14 days, and new data is deleted once it is older than 7 days.

See the following impact on End User Monitoring data:

Examples:

• config.appdata.shortterm.retention.days is changed from 7 days to 14 days:
  Old and new data is deleted once it is 14 days old.
• config.appdata.shortterm.retention.days is changed from 14 days to 7 days:
  Old and new data is deleted once it is 7 days old.

Synthetic Monitoring retention settings

Synthetic Monitoring has a separate data retention property, with the default of 7 days. Retention of both Synthetic test results and synthetic test result details are controlled by this setting.

kind: Core
metadata:
  name: instana-core
  namespace: instana-core
spec:
...
  properties:
    - name: config.synthetics.retention.days
      value: "7"
...

See the following impact on Synthetic Monitoring data:

Examples:

• config.synthetics.retention.days is changed from 7 days to 14 days:
  Data stored before the configuration change is deleted once it is older than 7 days, and data stored after the configuration change is deleted once it is older than 14 days.
• config.synthetics.retention.days is changed from 14 days to 7 days:
  Data stored before the configuration change is deleted once it is older than 14 days, and data stored after the configuration change is deleted once it is older than 7 days.

Applying your settings

After your configuration file for Core is ready, you can apply it as follows:

# Applay core.yaml configuration
kubectl apply -f path/to/core.yaml

# Check if all pods are running before proceeding
kubectl get pods -n instana-core

instana-console datastore host

For this purpose, the second host needs to be set up with a settings.hcl file as follows:

type      = "dual-clickhouse"
host_name = "<IP-accessible-from-the-k8s-cluster>"

dir {
  metrics    = "" // data dir for metrics
  traces     = "/mnt/traces" // data dir for log data
  data       = "/mnt/data" // data dir for any other data
  logs       = "/var/log/instana" // log dir
}

docker_repository {
  base_url = "artifact-public.instana.io"
  username = "_"
  password = "<Your-agent-key>"
}

Operator Core spec

On the operator side, the Core spec needs two Clickhouse entries and feature flag:

apiVersion: instana.io/v1beta2
kind: Core
metadata:
  name: instana-core
  namespace: core
spec:
...
clickhouseConfigs:
  - clusterName: local
    hosts:
      - <main DB hostname or IP>
    schemas:
      - application
  - clusterName: local
    hosts:
      - <second DB hostname or IP>
    schemas:
      - logs
  ...
featureFlags:
  - name: feature.logging.enabled
    enabled: true
...

Run and configure BeeInstana via instana-console host

To run BeeInstana as an additional Docker container on the instana-console database host, the following feature flag must be added to the file settings.hcl for new installations or to the file /root/.instana/effective-settings.hcl for existing installations.

...
feature "beeinstana" {
  enabled=true
}
...

After that, the change must still be applied to the system with an instana datastores update. After the restart, an additional container named aggregator should appear. This service must then be accessible via port 9998 from the Instana backend.

Note: Ensure that the free space is enough in the storage location for the metric data.

An estimated disk size of 1 million metrics can be derived from this table:

Now the BeeInstana aggregator must be exposed to the operator Core spec, and the following feature flags must be enabled.

apiVersion: instana.io/v1beta2
kind: Core
metadata:
  name: instana-core
  namespace: core
spec:
...
datastoreConfigs:
  beeInstanaConfig:
    hosts:
      - <main-db-hostname/ip>
...
featureFlags:
  - name: feature.beeinstana.infra.metrics.enabled
    enabled: true
  - name: feature.infra.explore.presentation.enabled
    enabled: true
...

Run and configure BeeInstana via Operator

For self-hosted Instana environments on Kubernetes with large metric loads, you are recommended to deploy BeeInstana by using the BeeInstana operator. See the documentation for using BeeInstana on Kubernetes.

In the Instana operator, the following changes must now be made to the Core spec.

apiVersion: instana.io/v1beta2
kind: Core
metadata:
  name: instana-core
  namespace: core
spec:
...
datastoreConfigs:
  beeInstanaConfig:
    hosts:
      - <beeinstana-operator-service-name>
    clustered: true
...
featureFlags:
  - name: feature.beeinstana.infra.metrics.enabled
    enabled: true
  - name: feature.infra.explore.presentation.enabled
    enabled: true
...

Configure external storage for Synthetics Monitoring

Before enabling Synthetics monitoring, external storage in the form of Cloud storage Buckets needs to be configured in the storageConfigs section in the Core spec.

Currently, GCloud and S3 or compatible Buckets are supported.

For more information on storageConfigs spec for synthetics, refer storageConfigs

Enable Synthetics components using feature flag

To enable Synthetics components, set the following feature flags under the featureFlags configuration in the Core spec.

featureFlags:
    ...
      - name: feature.synthetics.enabled
      enabled: true