Storage for Databand metadata

When you use self-hosted Databand, you must use both a PostgreSQL database and a Redis cache. The following sections describe how to manage storage for self-hosted Databand.

You can use an external PostgreSQL database for either Docker Compose or Kubernetes deployments, and in you can see specific storage options for Docker Compose and Kubernetes.

Docker compose: Internal Docker Compose storage
Kubernetes: Internal storage; External Redis instances; Setting a storageClass

Prerequisites

Make sure that your system has enough storage for your business needs. See the following table for the required amount of storage depending on the number of resource units (RU) that your business uses.

Note: The values in the table are relevant only for your PostgreSQL database. The required memory for your Redis instance is 50% of your PostgreSQL database memory.

Table 1. Minimum necessary database storage for different business sizes, based on RU usage
RU range	Database vCPU	Database memory (GB)	Database storage (GB)
1 - 200	2	3.75	80
201 - 1,000	2	7.5	220
1,001 - 3,000	4	15	660
3,001 - 8,000	8	30	1758
8,001 - 10,000	16	60	2198
10,001 - 15,000	32	120	3296

Creating an external PostgreSQL database

If you use Databand for production, follow these steps to create an external PostgreSQL database to manage your Databand metadata. The steps are the same for both Docker Compose and Kubernetes deployments. You must use PostgreSQL 16.x or later.

Connect to the PostgreSQL instance with the master user:

psql -h <RDS Postgress URL> -U postgres_user -d postgres

Create the Databand database and user:

create database databand;

create user databand with encrypted password 'databand';
grant all privileges on database databand to databand;
alter database databand owner to databand;

Use the master user to connect with the Databand database:

psql -h <RDS Postgres url> -U postgres_user -d databand

Create the uuid-ossp extension:

CREATE EXTENSION "uuid-ossp" SCHEMA pg_catalog

Fill in the following values in user-values.yaml. The sql_alchemy_conn is postgresql+psycopg2://username:password:host:port/dbname.

## user-values.yaml

postgresql:
  enabled: false

sql_alchemy_conn:
  protocol: "postgresql+psycopg2"
  username: "databand"
  password: "databand"
  host: "databand.example.com"
  port: "5432"
  dbname: "databand"

If you have an existing secret that has a password in the namespace, you can read from it by using the following example:

## user-values.yaml

postgresql:
  enabled: false

sql_alchemy_conn:
  protocol: "postgresql+psycopg2"
  username: "databand"
  password: ""
  host: "databand.example.com"
  port: "5432"
  dbname: "databand"
  existingSecret:
    name: "somesecretname"
    secretKeys:
      userPasswordKey: "password"

Creating internal storage for Docker Compose deployments

For all Docker Compose deployments, you must use the internal Redis instance, which exists by default upon deploying Databand with Docker Compose. No further action is required to create, enable, or use the internal Redis instance.

Databand Self-Hosted by default also contains a local PostgreSQL instance, which you can use if you prefer to use an internal storage option or if you can't use an external PostgreSQL instance. Use the following command to create the local PostgreSQL database:

COMPOSE_FILE=docker-compose.yml:mods/local_pg.yml:mods/monitoring.yml

Kubernetes deployments

If you use self-hosted Databand for production, use an external PostgreSQL database and an external Redis cache. If your organization's security policies prevent you from using external storage options, you can use internal storage instead.

Using internal storage options for Kubernetes

If you are unable to use an external storage option, or if you prefer to use an internal storage option, you can use the local PostgreSQL and Redis instances, which exist by default in self-hosted Databand. The following section describes how to use these internal options.

Create secure database connection credentials with:

kubectl create secret generic databand-postgres --namespace databand-system --from-literal=postgres-password=$(openssl rand -base64 13)
kubectl create secret generic databand-redis --namespace databand-system  --from-literal=redis-password=$(openssl rand -base64 13)

Update your user-values.yaml:

## user-values.yaml

postgresql:
  existingSecret: databand-postgres

redis:
  existingSecret: databand-redis

If you can't use publicly available images, for example for security reasons, you can set your own image properties in user-values.yaml as in the following example:

## user-values.yaml
databand:
  initdb:
    job:
      initContainers:
        wait_postgres:
          busybox:
            image:
              repository: <YOUR_REPOSITORY_FOR_BUSYBOX_IMAGE>
              tag: <YOUR_TAG_FOR_BUSYBOX_IMAGE>

postgresql:
  existingSecret: databand-postgres
  image:
    registry: <YOUR_REGISTRY_FOR_POSTGRES_IMAGE> # e.g. docker.io
    repository: <YOUR_REPOSITORY_FOR_POSTGRES_IMAGE> # e.g. bitnami/postgresql
    tag: <YOUR_TAG_FOR_POSTGRES_IMAGE> # e.g. 12.11.0

redis:
  existingSecret: databand-redis
  image:
    registry: <YOUR_REGISTRY_FOR_REDIS_IMAGE> # e.g. docker.io
    repository: <YOUR_REPOSITORY_FOR_REDIS_IMAGE> # e.g. bitnami/redis
    tag: <YOUR_TAG_FOR_REDIS_IMAGE> # e.g. 7.0.5-debian-11-r15

Using an external Redis cache

By default, deploying Databand Self-Hosted uses a local Redis instance. Follow these steps to create an external Redis instance to manage your Databand metadata. You must use Redis 6.x.

Create a Redis instance by using:

## user-values.yaml

redis:
  enabled: false

inmemorystore:
  host: "10.1.3.4"
  port: "6379"
  password: "verysecretpassword"

If you want to read your in-memory store password from an existing secret, specify the following values:

## user-values.yaml

redis:
  enabled: false

inmemorystore:
  host: "10.1.3.4"
  port: "6379"
  password: ""
  existingSecret:
    name: "somesecretname"
    secretKeys:
      passwordKey: "password"

Update your user-values.yaml file:

## user-values.yaml

postgresql:
  existingSecret: databand-postgres

redis:
  existingSecret: databand-redis

Setting a specific storageClass for Databand components

If your Kubernetes cluster doesn't have a default storageClass set, you must specify a storageClass for Databand components, which requires a provisioning of PersistentVolume objects.

To set this parameter, add the following values in user-values.yaml:

postgresql:
  global:
    storageClass: <YOUR_STORAGE_CLASS_NAME>

prometheus:
  server:
    persistentVolume:
      storageClass: <YOUR_STORAGE_CLASS_NAME>

redis:
  global:
    storageClass: <YOUR_STORAGE_CLASS_NAME>

webapp:
  persistence:
    storageClass: <YOUR_STORAGE_CLASS_NAME>