Smart Alerts for logs

With Smart Alerts for logs, you can automatically receive alerts when specific log messages occur more often than usual, or a known problem visible in the logs is regressing.

Instana suggests the thresholds and remaining configurations for you. When you add multiple alerting channels to the configuration, and Instana automatically creates a customized alert for you.

Adding an alert

To add an alert, complete the following steps:

  1. From the navigation menu in the Instana UI, select Logs.
  2. Select the Smart Alerts tab.
  3. Click Create Smart Alert.

The Create Smart Alert opens the alert configuration dialog where you can configure Smart Alerts.

The alert configuration process includes the following steps:

  1. Define the scope
  2. Define the threshold for violations
  3. Define the time threshold about when to be alerted
  4. Select the alert channels that are to be notified
  5. Define the alert properties
  6. Add custom payloads to be included in alerts

Defining the scope

In the scope section, the metric log count metric is selected by default. You can narrow down the scope by adding filters based on the log content or underlying infrastructure. The metric results can be grouped with the available grouping tags. Currently, multiple grouping tag is not supported in Log Smart Alerts.

Defining the scope
Figure 1. Defining the scope

Defining the threshold

Currently, Log Smart Alert supports only the static threshold option. Static thresholds do not change over time. A static threshold is set when you create or modify the Smart Alert. The threshold might stop being relevant after the underlying metric is changed significantly. You can select a threshold operator to define the threshold condition.

After the scope and threshold is defined, the chart is plotted based on the historic data against the metrics. The maximum of 7 days historic data are available for visualization in the chart. You can switch between the last 24 hours to 7 days of historic data to visualize the historic variations of metric data.

Based on the historic data and threshold conditions, the following image shows a chart that displays the alerts that might trigger with the current set threshold value:

Alerts Preview
Figure 2. Alerts Preview

If you select any grouping options, the grouping results might appear as a table just after the chart. To analyze the metric data trends in the chart against each grouping, select the respective rows in the table as displayed in the following image:

Grouping selection
Figure 3. Grouping selection

Defining the time threshold

For the alert that is triggered, you can add more conditions in the Time Threshold section when the defined threshold for the selected metric is violated.

The following typical conditions, often used in practice, are as follows:

  • Persistence over time: Select a time window and the number of consecutive times of violation as shown in the following image. You receive an alert when the metric violates a defined threshold over the defined time window.

Time threshold
Figure 4. Time threshold

Adding alert channels

You can configure different alert channels for both warning and critical severity level in Smart Alerts for Logs. To add alert channels, complete the following steps:

  1. Click Select Alert Channel.
  2. From the list of preconfigured channels, select the channels from which you want to receive the alerts.

If a threshold value is set for warning and critical severities, you can set the alert channels for each severity. If a threshold value is set for both severities, all the alert channels are selected for the warning severity by default.

The following image shows alert channels with both severities configured:

Time threshold
Figure 5. Alert channels with multiple severities

If a threshold value is set only for one severity, the severity is displayed for every alert channel as the Alert Level.

The following image shows alert channels with one severity configured:

Time threshold
Figure 6. Alert channels with one severity

For more information about creating channels, see Alert channels.

Selecting alert properties

Adding more alert properties is optional.

Adding more alert properties provides you with the additional configuration that best suits your needs. You can edit the current title and description of the alert, define the alert level (warning or critical) as shown in the following image:

Alert properties
Figure 7. Alert properties

Adding custom payloads

You can customize alert notifications by adding the following custom payloads:

  • Global custom payloads: These payloads are relevant in all alert notifications that are sent by Instana.
  • Alert-specific custom payloads: These payloads are relevant in alert notifications for a specific alert configuration that is sent by Instana.

An alert notification can include both global and alert-specific custom payloads (if applicable), but the alert-specific configuration is prioritized over the global configuration. As a result, if you use the same key, the value of the global custom payload field is overridden by the alert-specific one.

To add global custom payloads, see Configure custom payload globally.

The following image shows globally defined custom payloads that are used in the alert configuration:

Read only global custom payload
Figure 8. Read only global custom payload

To add alert-specific custom payloads, complete the following steps:

  1. Click Add Row in the Custom Payloads section.
  2. Enter a key to identify the custom payload entry.
  3. Select the value type of the custom payload: Static or Dynamic
  4. Define the value of the payload entry:

    1. For Static payload, enter the value.

    2. For Dynamic payload, click Select tag and choose a dynamic tag. You can use the suggestions to select the correct key for the selected dynamic tag or add it manually.

      The following image shows how to select a dynamic tag:

      Dynamic Custom Payload
      Figure 9. Dynamic custom payload

      The following image shows suggestions to select the correct key for the selected dynamic tag:

      Dynamic Custom Payload Suggestions
      Figure 10. Dynamic custom payload suggestions