Monitoring and logging by using any monitoring tool

IBM Fusion provides multiple ways to monitor the health of the hardware and software.

  • Events are fed into the Red Hat® OpenShift® event manager

    The following example IBM Fusion events for your reference:

    • Hardware events for IBM Fusion HCI System, including switches and nodes
    • Events that are related to the data services of IBM Fusion, such as backup and restore

    As IBM Fusion feeds events into Red Hat OpenShift event manager, they show up in the integration that you use to monitor OpenShift events.

  • Metrics are fed into Prometheus

    IBM Fusion feeds metrics into a Red Hat OpenShift Prometheus instance.

    The advantage is that they are available in any monitoring tool you are using with Prometheus, such as Grafana. IBM Fusion provides a set of default Grafana dashboards and an access link to the interface.

  • IBM Fusion provides logs for its services
  • IBM Fusion provides a user interface that shows the status of the system
  • IBM Fusion provides CRs that reflect the status of the system

Events and metrics provide observability that can be fed into other monitoring systems, such as Grafana and ELK.

Important:
  • IBM Fusion does not install Grafana and ELK as part of its installation.
  • If you already have monitoring tools that you use in your environment, then you can feed IBM Fusion observability into these existing tools. For example, if you use Grafana to monitor your OpenShift environment, then you can import metrics and dashboards of IBM Fusion into its instance.

Monitoring

The metrics are collected from storage, network, and compute components and sent to Prometheus, which is then propagated to Grafana for visualization.

Follow the steps to access IBM Fusion information in Grafana. After Grafana is installed and Prometheus data source is configured, you can import dashboards of IBM Fusion with the following steps:
Important:
  • You must have a working Grafana instance.
  • The OpenShift Container Platform version must be at 4.12 or later.
  • The following steps 1 - 5 explain how to configure Prometheus data sources. If it is already configured, then you can directly go to step 6.
Important: Ensure that you change the namespace or project before following the steps. Run the following command to change the namespace.
oc project <your-grafana-namespace>
  1. Create a ServiceAccount associated with your Grafana, and replace the namespace.
    oc create serviceaccount grafana -n <your-grafana-namespace>
  2. Create a Role Binding to a ServiceAccount that is associated with your data source in Grafana, and replace the namespace. So, that can use the Prometheus cluster.
    oc create clusterrolebinding grafana-cluster-monitoring-view \
  --clusterrole=cluster-monitoring-view \
  --serviceaccount=<your-grafana-namespace>:grafana
  3. Create a secret of type service-account-token by using the following sample YAML.
    apiVersion: v1
kind: Secret
type: kubernetes.io/service-account-token
metadata:
  name: grafana-secret
  annotations:
    kubernetes.io/service-account.name: grafana
  4. Run the following command to get a token.
    oc get secret grafana-secret -ojsonpath='{.data.token}' | base64 -d
  5. Configure Grafana with a Prometheus data source and authentication.
    1. Log in to Grafana user interface.
    2. From the menu, go to Configurations and select Data Sources.
    3. Click Add data sources and select Prometheus.
    4. Update the following values:
      URL
      Run the following command to get the address of the Cluster Prometheus Route.
      oc get route prometheus-k8s -n openshift-monitoring
      Enable Skip TLS Verify parameter under Auth section.
      Custom HTTP Headers
      HTTP Method: Post
      Header: Authorization
      Value: bearer <token-from-step-4>
  6. Download the JSON file for each of the dashboard:
  7. Import the files in the Grafana dashboard. For the procedure to import files in Grafana dashboard, see Importing files.
    Note: Select Upload JSON file option and upload JSON downloaded in step 6.
  8. Verify whether that Grafana dashboard is imported correctly.
  9. Click Dashboards menu and choose the newly imported dashboard.

Viewing dashboard

In Grafana, you can view different dashboards for storage, network, and compute wherein the following metrics are captured for each component and represented graphically:

The following metrics are captured for each of the components:
Storage Network Compute
Throughput High-speed switches Energy status
diskreadtime Management switches Thermal status
ReadIOPs Spine switches  
WriteIOPs    
Disk capacity    

An example Grafana screen that shows the networking dashboard of IBM Fusion for a high-speed switch.screnncaptureofsnmpexporter

An example Grafana screen that shows the storage dashboard of IBM Fusion:grafannaboardexample

For more information about Grafana, see https://grafana.com/docs/. For more information about Prometheus, see https://prometheus.io/docs/introduction/overview/.

Log and audit logs

Logs and audit logs are collected in ElasticSearch for IBM Fusion and viewed in Kibana. The IBM Fusion HCI System component logging and OpenShift Container Platform logs and audit logs can be viewed on Kibana. For more information about searching the logs, see https://docs.openshift.com/container-platform/4.15/logging/log_visualization/logging-kibana.html.

To open Kibana from IBM Fusion HCI System user interface, click the Kibana outbound link. As a prerequisite, you must create a basic index pattern. For the steps to create the basic index pattern, see Creating an audit index pattern and visualize audit data in Kibana.

To know more about Kibana, see https://www.elastic.co/guide/en/kibana/index.html.

To export OpenShift Container Platform logs from Kibana or Elasticsearch, see https://access.redhat.com/solutions/4599791.