Automated configuration

Starting with version 4.2 of the performance monitoring tool, sensors can be configured on nodes that are part of an IBM Spectrum Scale™ cluster through an IBM Spectrum Scale based configuration mechanism. However, this requires the installation of IBM Spectrum Scale 4.2 or later versions on all the nodes where a sensor is running and where the sensors are to be configured. It also requires the entire cluster to be at least running IBM Spectrum Scale 4.1.1 or later version, and the execution of the mmchconfig release=LATEST command.

The automated configuration method allows the sensor configuration to be stored as part of the IBM Spectrum Scale configuration. Automated configuration is only available for the sensor configuration files (/opt/IBM/zimon/ZIMonSensors.cfg) but not for the collector configuration files (/opt/IBM/zimon/ZIMonCollector.cfg). In this setup, the /opt/IBM/zimon/ZIMonSensors.cfg configuration file on each IBM Spectrum Scale node is maintained by IBM Spectrum Scale. As a result, the file must not be edited manually because whenever IBM Spectrum Scale needs to update a configuration parameter, the file is regenerated and any manual modifications are overwritten. Before using the automated configuration, an initial configuration needs to be stored within IBM Spectrum Scale. You can store this initial configuration by using the mmperfmon config generate command as shown:

prompt# mmperfmon config generate \
--collectors collector1.domain.com,collector2.domain.com,...

The mmperfmon config generate command will use a template configuration file for generating the automated configuration. The default location for that template configuration is /opt/IBM/zimon/defaults/ZIMonSensors.cfg.

The template configuration includes the initial settings for all the sensors and may be modified prior to invoking the mmperfmon config generate command. This file also includes a parameter called colCandidates. This parameter specifies the number of collectors that each sensor should report its data to. This may be of interest for high-availability setups, where each metric should be sent to two collectors in case one collector becomes unavailable.

Once the configuration file is stored within IBM Spectrum Scale, it can be activated as follows:
prompt# mmchnode --perfmon –N nodeclass1,nodeclass2,…

Note: Any previously existing configuration file is overwritten. Configuration changes result in a new version of the configuration file, which is then propagated through the IBM Spectrum Scale cluster at the file level.

To deactivate the performance monitoring tool, the same command is used but with the --noperfmon switch supplied instead. Configuration parameters can be changed with the following command where parami is of the form sensorname.sensorattribute:
prompt# mmperfmon config update param1=value1 param2=value2 …

Some sensors such as the cluster export services sensors are run on a specific set of nodes. Other sensors such as the GPFSDiskCap sensor are run on a single node in the cluster since the data reported is the same, independent of the node the sensor is running on. For these types of sensors, the restrict function is especially intended. For example, to restrict a NFSIO sensor to a node class and change the reporting period to once every 10 hours, you can specify NFSIO.period=36000 NFSIO.restrict=nodeclass1 as attribute value pairs in the update command.

Some sensors, such as VFS, are not enabled by default even though they have associated predefined queries with the mmperfmon query command. This is so because the collector might display performance issues of its own if it is required to collect more than 1000000 metrics per second. To enable VFS sensors, use the mmfsadm vfsstats enable command on the node. To enable a sensor, set the period value to an integer greater than 0 and restart the sensors on that node by using the systemctl restart pmsensors command.

Removing an automated configuration

When upgrading the performance monitoring tool, it is important to note how the previous version was configured and if the configuration mechanism is to be changed. Before IBM Spectrum Scale 4.2, the system was configured using a file-based configuration where the configuration files were manually edited and propagated to the requisite nodes. If the configuration mechanism is to be changed, it is important to verify that the installed versions of both IBM Spectrum Scale and the performance monitoring tool support the new configuration method. However, if you want to use the manual configuration method, then take care of the following:

None of the nodes in the cluster should be designated perfmon nodes. If the nodes in the cluster are designated as perfmon nodes then run mmchnode --noperfmon –N all command.
Delete the centrally stored configuration information by issuing mmperfmon config delete --all command.

The /opt/IBM/zimon/ZIMonSensors.cfg file is then maintained manually. This mode is useful if sensors are to be installed on non-Spectrum Scale nodes or if you want to have a cluster with multiple levels of IBM Spectrum Scale running.