Observability setup

Edit online

This section guides you through the installation and configuration of observability components in a Kubernetes cluster, including the necessary tools, and modifications.

Required components

  • An IBM Aspera Enterprise Webapps cluster on Kubernetes
  • A command line JSON (jq) and YAML (yq) processor
  • kubectl
  • Helm (kubernetes package manager)
  • Access to the Kubernetes cluster and appropriate permissions to create namespaces, deploy resources, and access configurations

Install the observability component

  1. Run the installation script:

    For default settings:

    ./opt/aspera/o11y/scripts/setup.sh -i
    For custom options:
    ./opt/aspera/o11y/scripts/setup.sh -i -f log.log -n aew-monitor -r aewmonitoring -p secure-password -c mysql,cassandra
  2. To uninstall the observability component:
    • Uninstall and delete namespace and Prometheus persistent storage: ./opt/aspera/o11y/scripts/setup.sh -u delete -f log.log
    • Uninstall but keep namespace and Prometheus persistent storage: ./opt/aspera/o11y/scripts/setup.sh -u keep -f log.log

Available options (setup.sh script)

Option Long format Description Default value
-h --help Display help message —
-i --install Install observability components for IBM Aspera Enterprise Webapps  
-u <MODE> --uninstall <MODE> Uninstall observability components with either delete or keep namespace keep

Installation options (requires -i / --install)

Option Long format Description Default value
-n <NAME> --namespace <NAME> Namespace for observability components aop-monitor
-r <NAME> --release_name <NAME> Prometheus Operator release name aopmonitoring
-s <NAME> --service_type <NAME> Specifies how Grafana service is exposed. Either NodePort or LoadBalancer NodePort
-p <STRING> --password <STRING> Grafana admin password prom-operator
-c <LIST> --components <LIST> Components to install (comma-separated) All
-sc <NAME> --storageClass <NAME> StorageClass name for the PVC required by Prometheus Cluster's default
-t <PERIOD> --retention <PERIOD> Set Prometheus data retention period in days —

General options

Option Long format Description Default value
-f <PATH> --file_name <PATH> Log file path and name Console
Namespace selection (-n)

You can install the observability component in either a new and dedicated namespace (best) or an existing namespace. When uninstalling, if you created a new namespace for observability, you can safely use -u delete to remove everything. If you chose to use an existing namespace, use -u keep to avoid removing other resources in that namespace.

Note: The install (-i) and uninstall option (-u) support an additional argument -f to specify a log file.

Component selection (-c)

The basic installation includes 28 kubernetes monitoring dashboards. You can add specialized dashboards for database observability

The available values for this argument are:

  • All (default): Installs all available dashboards.
  • mysql: Adds MySQL specific dashboards.
  • redis: Adds Redis specific dashboards.
  • cassandra: Adds Cassandra specific dashboards.
  • kafka: Adds Kafka specific dashboards.
  • spark: Adds Spark specific dashboards.
  • elasticsearch: Adds Elasticsearch metrics dashboard.
  • emissary: Adds Emissary and Ambassador dashboard.
  • a8-backend: Adds Analytics backend dashboard.

You can combine multiple components with commas and no space: ./scripts/setup.sh -c mysql,cassandra. This example will install the base Kubernetes dashboards plus MySQL and Cassandra dashboards.

Examples
  • Install with defaults: ./opt/aspera/o11y/scripts/setup.sh -i
  • Custom installation: ./opt/aspera/o11y/scripts/setup.sh -i -n observability -r prom-stack -p secure-password -c redis -f log.log

Monitoring with Grafana

Grafana helps visualize and analyze real-time metrics, allowing you to monitor system performance and troubleshoot issues.

Note: The following configuration applies to local Kubernetes clusters.
Access the Grafana interface

To access the Grafana UI, use the URL provided in the setup.sh script output, for example: https://<orgname>.<subdomain>.<domain>:3000. The exact configuration depends on the service type (LoadBalancer or NodePort) you selected during installation.

NodePort

If you chose the NodePort option, you must configure your external load balancer (or ingress component) to route traffic to Grafana. For example, with HAProxy, add the following configuration at the end of /etc/haproxy/haproxy.cfg:

frontend grafana
    mode tcp
    bind <orgname>.<subdomain>.<domain>:3000
    option tcplog
    use_backend grafana_http

backend grafana_http
    mode tcp
    balance roundrobin
    option tcp-check
    server worker1 <IP1>:32432 check
    server worker2 <IP2>:32432 check
    server worker3 <IP3>:32432 check
    server worker4 <IP4>:32432 check
To obtain the node port of the Grafana service run:
kubectl get service -A | grep grafana | sed -e "s|.*443:||g" -e "s|/.*||g"
Note: 32432 is used as a placeholder, replace it with the actual number obtained after running the previous command. Replace <IP1>, <IP2>, etc. with the IP addresses of your worker nodes.
Restart the HAProxy after updating the configuration:
systemctl restart haproxy
LoadBalancer

If you chose the LoadBalancer option, and your cluster is running on a supported cloud provider (such as AWS, GCP, or Azure), Kubernetes automatically provisions and manages the external load balancer. In this case, the route to Grafana is created automatically when deploying Grafana and no additional configuration is required.

Access on OCP clusters

For OpenShift clusters, the Grafana dashboard is available at https://monitoring.<subdomain>.<domain>.

Login credentials

Use the following credentials to log in:

  • Username: admin
  • Password: Either prom-operator (default) or your custom password if you used the -p argument during the installation.

Once logged in, navigate to the Dashboard section to access the preconfigured monitoring dashboards.

Find the dashboards
  1. On the left sidebar, click Dashboards.
  2. Navigate to the corresponding folder based on what you want to monitor.
  3. Select the specific dashboard to view the metrics.

For a list of dashboards installed with the basic package see the Reference: List of preinstalled dashboards section.