Smart Alerts for service level objectives

Permissions

You do not need permission to create Smart Alerts for SLOs. However, your ability to create alerts is limited by the permissions for any entity that is associated with your SLO, such as applications or websites. If you have permission to view an entity, you can create Smart Alerts for an associated SLO.

Creating Smart Alerts for SLOs

To create a Smart Alert for SLOs, complete the following steps:

1. From the Instana UI navigation menu, select Service Levels. The Service Levels page shows an overview of all SLOs and their status in alphanumeric order.

2. Click + ADD.

3. Click Add Smart Alert.

Alternatively, you can create a Smart Alert from a specific SLOs dashboard by navigating to that dashboard.

4. The Smart Alert configuration modal is displayed.

5. Select the type of SLO for which you want to create a Smart Alert.

6. Select one or multiple SLOs from the list for which you want to create Smart Alerts.

7. Select the type of blueprint to be alerted on:

   • Error budget: An alert is sent when the error budget usage reaches a certain percentage.

   • Status: An alert is sent when the SLO status is less than the configured threshold.

   • Burn rate: An alert is triggered based on the SLO (Service Level Objective) burn rate and the specified threshold. Currently, two types of burn rate alert configurations are supported

     • Single alert window: Uses a single threshold and a single alerting window to monitor the Error Budget Burn Rate
     • Long and short alert window: Uses two distinct alerting windows (long and short), each with its respective threshold to examine data over different time periods. The long alerting window looks back over an extended period to evaluate whether the burn rate has exceeded the threshold. It helps identify issues that develop gradually and might pose future problems if not addressed. In contrast, the short alerting window analyzes a shorter time frame to quickly detect if the burn rate rises sharply. This is useful for identifying sudden spikes in errors or downtime that might indicate immediate and critical issues. Using both windows to determine the burn rate allows for a more balanced and accurate alerting mechanism helping ensure that alerts are triggered only when both short-term volatility and long-term patterns indicate a sustained problem.

     Users can configure the durations and thresholds for both the long and short alert windows based on their monitoring needs. However, as a standard practice, the short alert window should typically be 1/12th the length of the long alert window to maintain meaningful contrast between short-term anomalies and long-term trends.

8. Select whether to configure the error budget, SLO status, or burn rate. For the burn rate option, define alert window(s) and threshold(s). For error budget and SLO status, only the threshold percentage needs to be set.

9. Set the time threshold condition that is to be met.

10. Click Select alert channels.

11. The list of available alert channels is displayed.

12. Select one or more alert channels. If no channels appear, create one by clicking Create alert channel.

13. Click Add.

14. Configure the alert properties:

• Title: Title for the Smart alert. Title is used to identify the Smart Alert.
• Alert level: Choose the style of the alert. You can set the alert to Critical or Warning.
• Triggers incident: Choose whether the Smart Alert triggers an incident or not.
• Description: Give the Smart Alert a description to provide more context.

Incident triggering is only supported for application entities.

15. Optional: Add custom payload to the Smart Alert by clicking Add Row and specify the key-value pairs for each added row.

16. Click Create.