Monitoring platform logs

IBM® Power® Virtualization Center version 2.1.0 provides monitoring features with open source components such as OpenSearch, Logstash and Filebeat, a leading stack of services that are designed to facilitate the capture, transformation, and filtering of large quantities of data. In addition, a separate component called OpenSearch Dashboards provides visualization and analysis capabilities to handle the large amount of data provided by platform logs, helping derive insights and troubleshooting PowerVC deployment and have continued smooth operation. With the OpenSearch as its foundation, you can collect logs, quickly diagnose, and troubleshoot any issues.

An Overview

PowerVC version 2.1.0 monitoring feature is developed over industry leading open source components OpenSearch, OpenSearch Dashboards, and Logstash. In addition, the collection of log data is performed by a lightweight service called Filebeat. These services work in tandem to collect the log data (Filebeat), transform and filter it (Logstash), store and index it (OpenSearch) and then visualize and analyze it (OpenSearch Dashboards).

Functional highlights

PowerVC Log Monitoring feature provides you a new experience on how to easily maintain and troubleshoot your platform. Here is a summary of the functional highlights.
  1. All PowerVC and OpenStack logs are collected and fed into the monitoring stack components. With this, you can easily transform this data or filter it to focus on the information that is most relevant for the operation of your environment.
  2. Common log preamble information is normalized and transformed into specific fields on the OpenSearch database that can be easily queried, filtered, or correlated with other fields or values during visualization and analysis on the user interface. PowerVC Log Monitoring provides basic samples to process and visualize platform log data and an extension mechanism for users to create their own filters and dashboards. Only the PowerVC-provided filters and dashboard samples are supported. Users may find additional help with the open source community to aid with creation and troubleshooting of their own custom built filters and dashboards.
  3. Users can easily fetch large quantities of data as monitoring can index data on a need basis.
  4. The ability to visualize large quantities of log data and quickly narrow down to relevant entries to help diagnose and troubleshoot problems, that use the state-of-the-art OpenSearch Dashboards user interface.
  5. Sample visualizations and dashboards are provided for you to import in your environment and use as-is or customize or expand them bases on your requirement.
  6. A mechanism for the long-term maintenance of this data and prune unnecessary indexes automatically is provided, based on two types of criteria: age of data, or space taken by the data on the file system. This is specially helpful with active systems where large quantity of data is present, to avoid filling up the available space on the system too quickly.
  7. Users can backup the log monitoring configuration and log data and then restore it, either on the same host or on a different host.
  8. After transformation and filtering rule changes, users can reset log collection and trigger the replay of log collection again from scratch. You can also compress important log data from /var/log and share with the IBM support team. They can reset and replay the same logs, which in practice reproduces the user’s log environment helping the support team to identify potential problems.
  9. Users can choose when to install the monitoring feature. It can be installed alongside the rest of the PowerVC product, or later, as needed.
  10. Uninstall and complete clean-up are also provided in case you want to stop using monitoring feature, or want to migrate monitoring components to another instance (with optional backup / restore capabilities).
  11. Finally, log monitoring can either be installed on single-node or multi-node configurations of PowerVC. During multi-node configurations, all components of the Monitoring Stack are installed on all controller nodes (but not on managed systems or endpoints such as NovaLink systems). The services on all nodes work together with data being replicated on 2 or more nodes, which in effect provides a (n-1) resiliency to failure. That means that in a 3-node configuration even if one of the nodes go down Log Monitoring will still work as intended. The access to the user interface is also protected by using a virtual IP address along with an HAProxy configuration that load-balances UI access among active UI nodes.

Additional functionality and information

For more information on OpenSearch and its services, see OpenSearch project documentation.