Creating an anomaly detection rule

Anomaly detection rules test the result of saved flow or event searches to search for unusual traffic patterns that occur in your network. Behavioral rules test event and flow traffic according to "seasonal" traffic levels and trends. Threshold rules test event and flow traffic for activity less than, equal to, or greater than a configured threshold or within a specified range.

Before you begin

To create anomaly detection rules on the Log Activity tab, you must have the Log Activity Maintain Custom Rules role permission.

To create anomaly detection rules on the Network Activity tab, you must have the Network Activity Maintain Custom Rules role permission.

To manage default and previously created anomaly detection rules, use the Rules page on the Offenses tab.

About this task

When you create an anomaly detection rule, the rule is populated with a default test stack, based on your saved search criteria. You can edit the default tests or add tests to the test stack. At least one Accumulated Property test must be included in the test stack.

By default, the Test the [Selected Accumulated Property] value of each [group] separately option is selected on the Rule Test Stack Editor page.

An anomaly detection rule tests the selected accumulated property for each event or flow group separately. For example, if the selected accumulated value is UniqueCount(sourceIP), the rule tests each unique source IP address for each event or flow group.

The Test the [Selected Accumulated Property] value of each [group] separately option is dynamic. The [Selected Accumulated Property] value depends on the option that you select for the this accumulated property test field of the default test stack. The [group] value depends on the grouping options that are specified in the saved search criteria. If multiple grouping options are included, the text might be truncated. Move your mouse pointer over the text to view all groups.

Procedure

  1. Click the Log Activity or Network Activity tab.
  2. Perform an aggregated search.

    You can add a property to the group by in a new historical search or select a property from the Display list on the current search page.

  3. On the search result page, click Configure Configure icon, and then configure the following options:
    1. Select a property from the Value to Graph list.
    2. Select time series as the chart type from the Value to Graph list
    3. Enable the Capture Time Series Data check box.
    4. Click Save, and then enter a name for your search.
    5. Click OK.
    6. Select last 5 minutes from the Time Range list, while you wait for the time series graph to load.
    You must have time series data for the property that you selected in the Value to Graph list to run a rule test on that accumulated property.
  4. From the Rules menu, select the rule type that you want to create.
    • Add Anomaly Rule
    • Add Threshold Rule
    • Add Behavioral Rule
  5. On the Rule Test Stack Editor page, in the enter rule name here field, type a unique name that you want to assign to this rule.
  6. To apply your rule by using the default test, select the first rule in the anomaly Test Group list.

    You might need to set the accumulated property parameter to the property that you selected from the Value to Graph list that you saved in the search criteria. If you want to see the result sooner, set the percentage to a lower value, such as 10%. Change last 24 hours to a lesser time period, such as 1 hour. Because an anomaly detection tests on aggregated fields in real time to alert you of anomalous network activity, you might want to increase or decrease events or flows in your network traffic.

  7. Add a test to a rule.
    1. To filter the options in the Test Group list, type the text that you want to filter for in the Type to filter field.
    2. From the Test Group list, select the type of test that you want to add to this rule.
    3. Optional: To identify a test as an excluded test, click and at the beginning of the test in the Rule pane. The and is displayed as and not.
    4. Click the underlined configurable parameters to customize the variables of the test.
    5. From the dialog box, select values for the variable, and then click Submit.
  8. To test the total selected accumulated properties for each event or flow group, disable Test the [Selected Accumulated Property] value of each [group] separately.
  9. In the groups pane, enable the groups you want to assign this rule to.
  10. In the Notes field, type any notes that you want to include for this rule, and then Click Next.
  11. On the Rule Responses page, configure the responses that you want this rule to generate.
    Learn more about rule response page parameters for anomaly detection rules:

    The following table provides the Rule Response page parameters if the rule type is Anomaly.

    Table 1. Anomaly Detection Rule Response page parameters
    Parameter Description
    Dispatch New Event Specifies that this rule dispatches a new event with the original event or flow, which is processed like all other events in the system. By default, this check box is selected and cannot be cleared.
    Offense Naming

    If you want the Event Name information to contribute to the name of the offense, select the This information should contribute to the name of the associated offense(s) option.

    If you want the configured Event Name to contribute to the offense, select the This information should set or replace the name of the associated offense(s).

    Note: After you replace the name of the offense, the name won't change until the offense is closed. For example, if an offense is associated with more than one rule, and the last event doesn't trigger the rule that is configured to override the name of the offense, the offense's name won't be updated by the last event. Instead, the offense name remains the name that is set by the override rule.
    Severity The severity level that you want to assign to the event. The range is 0 (lowest) to 10 (highest) and the default is 5. The Severity is displayed in the Annotations pane of the event details.
    Credibility The credibility that you want to assign to the log source. For example, is the log source noisy or expensive? Using the list boxes, select the credibility of the event. The range is 0 (lowest) to 10 (highest) and the default is 5. Credibility is displayed in the Annotations pane of the event details.
    Relevance The relevance that you want to assign to the weight of the asset. For example, how much do you care about the asset? Using the list boxes, select the relevance of the event. The range is 0 (lowest) to 10 (highest) and the default is 5. Relevance is displayed in the Annotations pane of the event details.
    Ensure that the dispatched event is part of an offense

    As a result of this rule, the event is forwarded to the magistrate. If an offense exists, this event is added. If no offense was created on the Offenses tab, a new offense is created.

    Notify Events that generate as a result of this rule are displayed in the System Notifications item in the Dashboard tab. If you enable notifications, configure the Response Limiter parameter.
    Send to Local SysLog
    Select this check box if you want to log the event or flow locally. By default, the check box is clear.
    Note: Only normalized events can be logged locally on a QRadar® appliance. If you want to send raw event data, you must use the Send to Forwarding Destinations option to send the data to a remote syslog host.
    Add to Reference Set

    Adds events that are generated as a result of this rule to a reference set. You must be an administrator to add data to a reference set.

    To add data to a reference set, follow these steps:

    1. From the first list, select the property of the event or flow that you want to add.
    2. From the second list, select the reference set to which you want to add the specified data.
    Add to Reference Data

    To use this rule response, you must create the reference data collection.
    Remove from Reference Set

    If you want this rule to remove data from a reference set, select this check box.

    To remove data from a reference set, follow these steps:

    1. From the first list, select the property of the event or flow that you want to remove.
    2. From the second list, select the reference set from which you want to remove the specified data.
    Remove from Reference Data

    To use this rule response, you must have a reference data collection.

    Execute Custom Action You can write scripts that do specific actions in response to network events. For example, you might write a script to create a firewall rule that blocks a particular source IP address from your network in response to repeated login failures.

    Select this check box and select a custom action from the Custom action to execute list.

    You add and configure custom actions by using the Define Actions icon on the Admin tab.
    Publish on the IF-MAP Server If the IF-MAP parameters are configured and deployed in the system settings, select this option to publish the offense information about the IF-MAP server.
    Response Limiter Select this check box and use the list boxes to configure the frequency with which you want this rule to respond
    Enable Rule Select this check box to enable this rule. By default, the check box is selected.

    An SNMP notification might resemble: 

    "Wed Sep 28 12:20:57 GMT 2005, Custom Rule Engine Notification -
 Rule 'SNMPTRAPTst' Fired. 172.16.20.98:0 -> 172.16.60.75:0 1, Event Name:
 ICMP Destination Unreachable Communication with Destination Host is
 Administratively Prohibited, QID: 1000156, Category: 1014, Notes:
 Offense description"

    A syslog output might resemble: 

    Sep 28 12:39:01 localhost.localdomain ECS:
 Rule 'Name of Rule' Fired: 172.16.60.219:12642
 -> 172.16.210.126:6666 6, Event Name: SCAN SYN FIN, QID:
 1000398, Category: 1011, Notes: Event description
  12. Click Next.
  13. Click Finish.