Configuring performance monitoring options in GUI

You need to configure and enable the performance monitoring for GUI to view the performance data in the GUI.

Enabling performance tools in management GUI

The performance tool consists of sensors that are installed on all nodes that need to be monitored. It also consists of one or more collectors that receive data from the sensors. The GUI expects that a collector runs on a GUI node. The GUI queries the collectors for performance and capacity data. The following steps use the automated approach to configure and maintain performance data collection by using the mmperfmon CLI command. Manually editing the /opt/IBM/zimon/ZIMonSensors.cfg file is not compatible with this configuration mode.
  1. Install the necessary software packages. Install the collector software package, gpfs.gss.pmcollector, on all GUI nodes. Install the sensor software packages, gpfs.gss.pmsensors, on all nodes, which are supposed to send the performance data.
  2. Initialize the performance collection. Use the mmperfmon config generate --collectors [node list] command to create an initial performance collection setup on the selected nodes. The GUI nodes must be configured as collector nodes. Depending on the installation type, this configuration might be already completed before. However, verify the existing configuration.
  3. Enable nodes for performance collection. You can enable nodes to collect performance data by issuing the mmchnode --perfmon -N [SENSOR_NODE_LIST] command. [SENSOR_NODE_LIST] is a comma-separated list of sensor nodes' hostnames or IP addresses and you can also use a node class. Depending on the type of installation, nodes might be configured for performance collection.
  4. Review peer configuration for the collectors. The mmperfmon config update command updates the multiple collectors with the necessary configuration. The collector configuration is stored in the /opt/IBM/zimon/ZIMonCollector.cfg file. This file defines the collector peer configuration and the aggregation rules. If you are using only a single collector, you can skip this step. The GUI must have access to all data from each GUI node. For more information, see Configuring multiple collectors in IBM Storage Scale: Administration Guide.
  5. Review aggregation configuration for the collectors. The collector configuration is stored in the /opt/IBM/zimon/ZIMonCollector.cfg file. The performance collection tool is configured with predefined rules on how data is aggregated when it gets older. By default, four aggregation domains are created as shown:
    • A raw domain that stores the metrics uncompressed.
    • A first aggregation domain that aggregates data to 30-second averages.
    • A second aggregation domain that stores data in 15-minute averages.
    • A third aggregation domain that stores data in 6-hour averages.

    You must not change the default aggregation configuration as the already collected historical metric information might get lost. You cannot manually edit the /opt/IBM/zimon/ZIMonCollector.cfg file in the automated configuration mode.

    In addition to the aggregation that is done by the performance collector, the GUI might request aggregated data based on the zoom level of the chart. For more information, see Configuring multiple collectors in IBM Storage Scale: Administration Guide.

  6. Configure the sensors. Several GUI pages display performance data that is collected with the help of performance monitoring tools. If data is not collected, the GUI shows the error messages like "No Data Available" or "Objects not found" in the performance charts. Installation by using the ESS installer manages the default performance monitoring installation and configuration. The GUI help that is available on the various pages shows performance metric information. The GUI context-sensitive help also lists the sensor names.

    The Services > Performance Monitoring page provides option to configure the sensor configuration and provides hints for collection periods and restriction of sensors to specific nodes.

    You can also use the mmperfmon config show command in the CLI to verify the sensor configuration. Use the mmperfmon config update command to adjust the sensor configuration to match your needs. For more information, see Configuring multiple collectors in IBM Storage Scale: Administration Guide.

    The local file /opt/IBM/zimon/ZIMonSensors.cfg can be different on every node and the system might change this path whenever a configuration change occurs. Therefore, this file must not be edited manually when you are using the automated configuration mode. During distribution of the sensor configuration, the restrict clause is evaluated and the period for all sensors is set to 0 in the /opt/IBM/zimon/ZIMonSensors.cfg file. The setting is defined for those nodes that did not match the restrict clause. You can check the local file to confirm that a restrict clause worked as intended.

Configuring capacity-related sensors to run on a single-node

Several capacity-related sensors must run only on a single node as they collect data for a clustered file system. For example, GPFSDiskCap, GPFSFilesetQuota, GPFSFileset and GPFSPool.

It is possible to automatically restrict these sensors to a single node. For new installations, capacity-related sensors are automatically configured to a single node where the capacity collection occurs. An updated cluster, which was installed before ESS 5.3.7 (IBM Storage Scale 5.0.5), might not be configured to use this feature automatically and must be reconfigured. To update the configuration, you can use the mmperfmon config update SensorName.restrict=@CLUSTER_PERF_SENSOR command, where SensorName values include GPFSFilesetQuota, GPFSFileset, GPFSPool, and GPFSDiskCap.

To collect file system and disk level capacity data on a single node that is selected by the system, run the following command to update the sensor configuration.
mmperfmon config update GPFSDiskCap.restrict=@CLUSTER_PERF_SENSOR
If the selected node is in the DEGRADED state, then the CLUSTER_PERF_SENSOR is automatically reconfigured to another node that is in the HEALTHY state. The performance monitoring service is restarted on the previous and currently selected nodes.
Note: If the GPFSDiskCap sensor is frequently restarted, it can negatively impact the system performance. The GPFSDiskCap sensor can cause a similar impact on the system performance as the mmdf command. Therefore, to avoid using the @CLUSTER_PERF_SENSOR for any sensor in the restrict field of a single node sensor until the node stabilizes in the HEALTHY state, it is advisable to use a dedicated healthy node. If you manually configure the restrict field of the capacity sensors then you must ensure that all the file systems on the specified node are mounted to record file system-related data, like capacity.

Use the Services > Performance Monitoring page to select the appropriate data collection periods for these sensors.

For the GPFSDiskCap sensor, the recommended period is 86400, which means once per day. As the GPFSDiskCap.period sensor runs mmdf command to get the capacity data, it is not recommended to use a value less than 10800 (every 3 hours). To show fileset capacity information, it is necessary to enable quota for all file systems where fileset capacity must be monitored. For more information, see the -q option in the mmchfs command and mmcheckquota command.

To update the sensor configuration for triggering an hourly collection of capacity-based fileset capacity information, run the mmperfmon command as shown in the following example, 
mmperfmon config update GPFSFilesetQuota.restrict=@CLUSTER_PERF_SENSOR gui_node GPFSFilesetQuota.period=3600

Verifying sensor and collector configurations

Do the following to verify whether collectors are working properly:
  1. Issue systemctl status pmcollector on the GUI node to confirm that the collector is running. Start collector it if it is not started already.
  2. If you cannot start the service, verify the log file that is located at the following location to fix the issue: /var/log/zimon/ZIMonCollector.log.
  3. Use a sample CLI query to test if data collection works properly. For example,
    mmperfmon query cpu_user
Do the following to verify whether sensors are working properly:
  1. Confirm that the sensor is configured correctly by issuing the mmperfmon config show command. This command lists the content of the sensor configuration that is at the following location:/opt/IBM/zimon/ZIMonSensors.cfg. The configuration must point to the node where the collector is running and all the expected sensors must be enabled. An enabled sensor has a period greater than 0 in the same config file.
  2. Issue systemctl status pmsensors to verify the status of the sensors.