Promote alerts to an incident

About this task

In this example, all high priority incidents that include alerts at or higher then a defined business criticality level are automatically assigned to the Db2 admin group with a notification sent to the group's team leader of such incidents.

There are three preset business criticalities available in Cloud Pak for AIOps:

Note: Preset business criticalities can be edited after they are added to your environment. For more information, see Defining business criticality.

Example

1. Click the navigation icon at the upper-left corner of the screen to go to the main navigation menu.

2. In the main navigation menu, click Operate > Automations.

3. Click Create policy.

4. Click the Promote alerts to an incident tile.

5. Enter a name in Policy name, for example, "Assign high priority Db2 incidents to Db2 admins". You can also add an explanation of the policy in Description to help you and others understand the purpose of the policy, for example, "Automatically assign any priority 1 incidents from Db2 hosts to the Db2 admin group, and notify team leader".

6. Set the Execution order to 1.

7. Define how the policy is triggered to create an incident. Once triggered, it will look for alerts that match the conditions you specify in order to take action. The policy triggers are when an alert is created, updated, or both. Updated means the alert state changes. For example, if an alert changes from Severity 1 to Severity 5. In this example, select An alert is created.

8. Define the following conditions for alerts that will trigger the creation of an incident:

   1. Click Add condition and select Alert property.
   2. From the Property drop-down list, select alert.severity. You can type "sev" and the system will show in the property drop-down list all alert properties that contain the text "sev", which in this case is only alert.severity. From the Operator drop-down list, select greater or equal. From the Matches drop-down list, select only. In the Values field, select 5-Major.
   3. Click Add condition and select Business criticality.
   4. The Property field contains the business criticality value by default. From the Operator drop-down list, select greater or equal. From the Matches drop-down list, select only. In the Values field, select 65-Tier 2.

   

9. Set hold off time. Delaying the creation of an incident provides time for data about the issue to be ingested. This can allow the system to make a better decision about what incidents should be created. For example, for flapping alerts or alerts which might be resolved by an automated runbook. In these cases, the incident would only be opened if the alert remains open beyond the hold off time. Eliminating incidents from the UI which ultimately are not actionable.

   Enter the number of seconds that you want to wait before the incident is created (maximum 600 seconds). In this example, the hold off time is 10 seconds.

10. Set priority. Give a priority level to the incident that is created when the policy conditions are met. For this example, select Priority 1.

11. Select how you want to receive Notifications about the incident (optional):

    • Notify users in an existing ChatOps channel.
    • Notify users via email. To use this option, you must have an email notification integration set up.

12. Open a ticket (optional). You can multi-select ticketing options from the drop-down menu. The required connections for GitHub, ServiceNow, or Jira must be set up in Integrations.

13. Assign a user group and owner to route the incident to (optional). In this example, the user group is the Db2 admin group, and the owner the group's team leader.

    Example incident policy actions:

    

14. Click Create policy.

New and updated policies can take up to 2 minutes to take effect.