Customizing the OMNIbus probe rules file

Increasing the severity of alarms raised on service-impacting metrics

You can customize the default probe rules file to increase the severity of certain alarms. For example, some metrics that are monitored by Predictive Insights have a direct user impact. Such metrics include service response time and service availability. To increase the severity of these alarms, update the following section of the probe rules file:

        # Compute alarm severity depending on whether the potentially anomalous KPI(s)
        # contain a service-impacting KPI.
        # You will need to update this statement to reflect your TASP implementation.
        if (int($TASPCorrelationId) >= 0)  {
                if (regmatch($TASPAnomalousMetrics, Service_Impacting_KPI_1
                  || regmatch($TASPAnomalousMetrics, Service_Impacting_KPI_2)
                    ) {
                        @Severity = 4
                } else {
                        @Severity = 3
                }
        } else if (int($TASPCorrelationId) == -1)  {
                # Consolidated alarms - many problems in one alarm.
                @Severity = 4
        } else if (int($TASPCorrelationId) == -2)  {
                # Data Availability alarms
                @Severity = 5
        } else if (int($TASPCorrelationId) == -3)  {
                # Information Events should have a lower severity.
                @Severity = 2
        } else if (int($TASPCorrelationId) == -4)  {
                # System health alarms are very important.
                @Severity = 5
        }

        # Map the data source severity directly, if it exists.
        # Note: resolution events should be set to severity 1, the generic clear will 
					set them to 0 later
        update(@Severity)

The $TASPAnomalousMetrics element contains the list of metrics that are detected as anomalous by Predictive Insights. You can use the $TASPAnomalousMetrics element to set the severity of the alarm.

You can replace Service_Impacting_KPI_1 and Service_Impacting_KPI_2 by the full metric names, as configured by the mediation and visible in the Predictive Insights User Interface. These include the metric name but not the resource or node name.

You can also set the severity by using the resource names in $TASPMetricResourceNameList. You can construct a $TASPAnomalousMetricResourceNameList for this purpose. Follow the example for $TASPAnomalousMetrics in the default probe rules file.

You must tune your probe rule file according to the configuration of your metrics and groups. For example, if you have metrics with the same name in multiple metric groups, setting the severity on individual metrics can set the severity on a metric of the same name in an incorrect group.

Setting a custom alarm summary

By default, Predictive Insights provides a summary that shows the anomalous metric, its trend and actual and expected values. You can change the details in the probe rules file. For example, you can use the names of the anomalous metrics or resources:

        # Set a custom summary for alarms impacting a specific service
        if (regmatch($TASPAnomalousMetrics, Service_Impacting_KPI_1)) {
                @Summary = Anomaly detected on Service 1
        } else {
                @Summary = $Summary
        }
        update(@Summary)

Predictive Insights can generate alarms with several metrics. Therefore, metrics from two different services can be included in a single alarm. You can decide how you want to handle the scenario, for example, by adding both service names in the summary.

Setting alarm severity based on attributes present in the event

You can set the severity of an alarm based on a particular attribute in the alarm. The following example sets the alarm severity to Critical for any alarm that contains an Infrastructure attribute.

if (regmatch($TASPAttributeNames, "Infrastructure"))
{
@Severity = 5
}
update(@Severity)