Log anomaly alerts

Types

For log anomaly alerts, the Type attribute shows which of the log anomaly models detected the anomaly:

• Natural language anomaly
• Statistical baseline anomaly

Summaries

The Summary attribute of an alert provides more detail about why an anomaly was detected.

A summary might look similar to the following example:

Logs containing errors for component: QMA with a primary message code of AMQ9665E. Evidence includes: domain-specific entities.

• The Evidence includes statement can show the following values:

  • domain-specific entities, for example, message codes
  • embeddings (statistical baseline)
  • error entities, for example, HTTP error codes and exceptions
  • patterns + embeddings (natural language)

• A primary message code, such as AMQ9665E, is included only for alerts where an anomaly is detected in logs that originate from a domain-specific source, such as IBM MQ or WebSphere.

Categorization of log anomaly alerts

During the training phase, templates are extracted based on the input logs. Models are trained to represent the normal behavior of a component or service in an incident-free window. In the inference, any incoming new live log data is matched against the templates that were extracted during training, and the distribution for a fixed window is computed. Any new logs, error logs, or deviation from the normal distribution from training might be flagged as an anomalous window and generate a log anomaly event.

A template is classified as an error by using keyword matching with a predefined error dictionary and error symptom words.

Alerts that are generated for log anomalies come in two types:

• Alerts with the Logs containing errors for prefix are log anomaly alerts that are generated when an error template is included in the alert. Templates include internal lists of error keywords. For example, a template can be classified as Logs containing errors for when it includes the keywords expires, failure, or incomplete.
• Alerts with the Abnormal behavior in the logs for prefix are log anomaly alerts that are generated when no error templates are included in the alert.

Explainability

An alert is generated when a log anomaly is detected. The metadata for an alert contains fields and information that can help you understand the identified log anomaly.

See the following alert field descriptions to better understand reported log anomalies:

• description: If error templates were found in the ten-second window and if the count of these templates is greater than 0, these templates are captured in the description field. If error templates are not present after an internal ranking algorithm is applied, the top five templates that have a count value greater than 0 are included in this field.
• details: This field includes the text_dict field, which contains critical fields, such as template_ids and template_list. This field includes the sorted templates and entities based on an internal ranking algorithm that corresponds to the log anomaly. This ranking algorithm prioritizes error templates and low occurrence error templates. It also prioritizes error entities over other types of entities. Based on the ranking, only the top 5 templates are shown.

Microsoft Teams and Slack integrations

If a Microsoft Teams or Slack integration is set up, the alerts are also sent to those integrations. The same alert data is parsed and shown in Microsoft Teams or Slack.

• The Microsoft Teams or Slack UI parses and displays the templates that are associated with the alert.
• Use the preview logs feature to see a sample of the logs that are associated with a template.
• When you click Attach log message patterns on the Microsoft Teams or Slack UI, you receive all anomalous and normal logs in the ten-second window time range for the resource, also known as the application component, that is associated with the alert.
• For more information, see Configuring Slack applications for integration and Creating a Microsoft Teams integration.