This is the third in a series of articles on best practices for writing rules for ODM Decision Server Insights. Please start with the article on the Basics of Rule Writing, and then move onto Intermediate Rule Writing.
The first article covered when/occurs reactive rules, fired when an event is received, with optional event and entity conditions. The second article covered time-sensitive rules that use the scheduler to fire sometime after an event is received, or fire periodically.
This article expands upon those foundations to cover agents and rules that look for sophisticated patterns of events in the event history associated with an entity.
First let’s recap some key terminology and concepts.

Bound Entity

The bound entity for a Rule Agent is used to determine where the Rule Agent runs. A Rule Agent runs on the JVM that hosts the primary copy of the bound entity. By default there will be a synchronous replica and an asynchronous replica of the bound entity on other JVMs for High Availability. A Rule Agent has full read/write access to its bound entity.

Event History

When a Rule Agent runs it has read-only access to the event history associated with the bound entity. The runtime automatically manages the event history, adding events correlated to the bound entity to the history and removing old events from the event history when they are older than the time horizon specified by the agent descriptor for the rule agent.

Current Event

The Rule Agent has read-only access to the current event that has just been received. All events within ODM Decision Server Insights are immutable. Note that the current event is not necessarily the most recent event (based on the timestamp of the event), as events can (and often do!) arrive out of order.

Pattern Matching

A Rule Agent can detect patterns in the events in the event history, optionally using data from the current event and the bound entity. These patterns can be relatively simple, or can include sophisticated time and data constraints. Let’s look at some examples from the InsightsStarter sample projects.

Counting Events in the Event History: Silver Loyalty Status

when a purchase occurs, called LAST
if
    the customer status of 'the customer' is BRONZE
        and there are at least 5 purchases after 30 days before LAST
then
    set the customer status of 'the customer' to SILVER;
    emit a new customer notification where
        the customer is 'the customer' ,
        the message is "Congratulations you are a silver customer.";

This rule is relatively simple, it is counting the number of purchases in the event history, and if there are at least 5 within the 30 days before the timestamp of the current event, and the customer is BRONZE, then the customer moves up to the SILVER loyalty tier.
Note that in this case the current event has been explicitly named LAST. Naming the current event is usually optional; if it is not named then it is referred to as ‘this purchase’.

Using Definitions and Variable Bindings: Small Purchase Following Change of Address

when a purchase occurs
definitions
    set 'recent cod' to a change of address
        where this change of address is after 4 hours before this purchase ;
if
    the total amount of the order items of this purchase is between 200 and 1000
    and (the risk status of 'the customer' is NONE)
then
    set the risk status of 'the customer' to LOW ;
    emit a new alert where
        the customer is 'the customer' ,
        the message is "Low risk purchase: small purchase following a change of address";

This rule steps up the level of sophistication and is looking for a change of address event in the event history that has occurred within 4 hours of the purchase just received. If the purchase just received is between $200 and $1000 then the purchase is assigned a risk rating of LOW.
Note that here we are using a definitions clause to create a variable binding. This looks superficially similar to a local variable declaration in a procedural programming language but it functions very differently. Understanding how variable bindings work is the key to creating powerful pattern matching rules. In the example above the ‘recent cod’ variable binding will match all instances of change of address events in the event history whose timestamp is after 4 hours before the timestamp of the purchase event just received. So if the customer has changed their address 3 times in the past 4 hours and then made a purchase, the ‘recent cod’ binding will be evaluated 3 times. No need for iteration or looping!
Given the event history:

PURCHASE 1 for $300: timestamp is 3.30 PM (current event)
COD 3: timestamp is 3 PM
COD 2 : timestamp is 1 PM
COD 1: timestamp is 2 PM

The ‘recent cod’ binding will first match COD 3. Given that the purchase just received is for $300, and the customer risk status is NONE, then the customer risk status is set to LOW and an event is emitted.
The ‘recent cod’ binding will then match COD 2. The purchase amount condition will be satisfied, but the customer risk status condition is no longer NONE and the rule does not fire. The bindings for COD 1 will similarly not cause the rule to fire.
To test the rule we create an event sequence called “small purchase following change of address”:

using definitions from "globals";
emit a new change of address where
    the customer is the customer identified by 'PADDINGTON ID' ,
    the new address is a new address where
    the country is "France",
    the timestamp is 12/10/2015 11:22:10 AM;
define 'purchase' as a new purchase where
    the customer is the customer identified by 'PADDINGTON ID';
    
add 'EXPENSIVE ORDER ITEM' to the order items of purchase ;
add 'EXPENSIVE ORDER ITEM 2' to the order items of purchase ;
emit purchase , time-stamped 10 minutes later;

And then a test scenario that submits the event sequence and checks the state of the customer:

using definitions from "globals";
 
submit events from "register paddington";
check that the customer identified by 'PADDINGTON ID' exists;
submit events from "small purchase following change of address";
check that for the customer identified by 'PADDINGTON ID' :
    - the risk status of this customer is LOW;

The Inspector recording illustrates the power of the rule-based approach, with both the bronze loyalty status rule and the small purchase following change of address rule firing.

Another key concepts for variable bindings is that if the variable cannot be bound to anything (e.g. there are no change of address events in the event history), then the rule conditions are not evaluated. It is important to bear this in mind as you define your patterns. In short, variable bindings are more like queries over event history, with optional constraints, than procedural variable declarations.

Collecting Events from Event History: Online Purchase from a New Country

when an online purchase occurs , called ONLINE
definitions
    set 'recent online purchases' to all online purchases after 3 months before ONLINE
        where the source ip country of each online purchase is the source ip country of ONLINE
            and the source ip country of each online purchase is not the country of the home address of 'the customer';
if
    there are less than 2 online purchases in 'recent online purchases'
    and (the risk status of 'the customer' is NONE or the risk status of 'the customer' is LOW)
then
    set the risk status of 'the customer' to MEDIUM ;
    emit a new alert where
        the customer is 'the customer' ,
        the message is "Medium risk purchase: purchase from a new country." ;

This rule take the concept of variable bindings a step further and introduce more constraints on the bindings. In this case we bind ‘recent online purchases’ to be a collection of all online purchase events in event history that are timestamped after 3 months before ONLINE, where the source ip country of the online purchase is the same as the purchase event just received, and is not the home country of the customer. I.e. we now have a dynamic collection of all online purchase events. Again, if there are no online purchase events in the event history that match the constraints specified, then the variable binding will fail and the conditions will not be evaluated.
The if condition of the rule then checks that there are less than 2 elements in ‘recent online purchases’, and increases the customer’s risk status to MEDIUM if it was previously NONE or LOW. If there are less than 2 elements, there could be 1 elements or 0 elements, however we’ve already seen that if there were 0 elements then the binding has failed and the condition would not be evaluated. There must therefore be 1 element in the list for the condition to be true.
So, given a customer whose home country is UK, with a risk status of NONE.
And this event history:

Online Purchase OP1 : source IP France (the current event - ONLINE)

The variable ‘recent online purchases’ will be bound to [OP1]. There is 1 element in the collection, so the rule will fire, setting the risk status to MEDIUM.
If the customer makes another online purchase from Belgium, their event history would look like:

Online Purchase OP2 : source IP Belgium (the current event - ONLINE)

Online Purchase OP1 : source IP France

The variable ‘recent online purchases’ will be bound to [OP2], but the rule will not fire because the customer’s risk status is already MEDIUM.
If the customer now makes a second purchase from France:

Online Purchase OP3 : source IP France (the current event - ONLINE)

Online Purchase OP2 : source IP Belgium

Online Purchase OP1 : source IP France

The variable ‘recent online purchases’ will be bound to [OP3,OP1], but the rule will not fire because the collection contains more than 1 element. The logic here being that the customer has already recently made an online purchase from France so we shouldn’t identify the purchase as risky.

Constraints on Variable Bindings: fast foreign online purchase

when an online purchase occurs , called ONLINE
definitions
    set 'previous online purchase' to an online purchase
        where the period between this online purchase and ONLINE is shorter than 2 hours
            and the source ip country of this online purchase is not the source ip country of ONLINE
            and the source ip country of this online purchase is not the country of the home address of 'the customer';
if
    the risk status of 'the customer' is not HIGH
    and the number of online purchases after 'previous online purchase' is 1
then
    set the risk status of 'the customer' to HIGH ;
    emit a new alert where
        the customer is 'the customer' ,
        the message is "High risk purchase: 2 online purchases from different source IP countries within 2 hours." ;

 
This rule uses a variable binding to an online purchase event from the event history. It constrains the binding to only match online purchases that have occurred within 2 hours of the online purchase just received (the current event), and checks that the source IP country of the online purchase bound is not the same as the source IP country of the event just received, and that it is not the home country of the customer. The condition then checks the risk status of the customer and if is not HIGH increases their risk status and emits an event.
Let’s consider a customer living in the UK with this event history:

OP 3: country France (current event - ONLINE), 2 PM

OP 2: country UK, 1 PM

OP 1: country Belgium, 12.30 PM

The ‘previous online purchase’ binding will first attempt to bind to OP 3.
This binding will fail because there is an implicit constraint on variable bindings that they all bind to different events in the event history, and because the current event is already bound to ONLINE, the ‘previous online purchase’ binding fails.
The ‘previous online purchase’ binding will then attempt to bind to OP 2.
This binding will also fail because the country of OP 2 is the home country of ‘the customer’.
The ‘previous online purchase’ binding will then attempt to bind to OP 1.
This binding will succeed because it is not the current event (ONLINE), it is from a country that is not the home country of the customer, and it is from a different country than ONLINE.
The rule will not fire however, even is the risk status of the customer is not HIGH, because there are 2 events after OP 1: [OP 2, OP 3]. I.e. the rule as coded is looking for 2 consecutive online purchase events from different countries that are not the home country of the customer. The fact that we have OP 2 from the home country of the customer between OP 1 and OP 3 prevents the rule from firing.
If we remove OP 2 from the event history, we get:

OP 2: country France (current event - ONLINE), 2 PM

OP 1: country Belgium, 12.30 PM

The ‘previous online purchase’ binding will first attempt to bind to OP 2. This binding fails because it is the same as ONLINE.
The second binding to OP 1 will succeed because it is not the current event (ONLINE), it is from a country that is not the home country of the customer, and it is from a different country than ONLINE.
The rule will now fire because there is only 1 event after OP 1: [OP 2]. The action of the rule will set the risk status of the customer to HIGH and emit an alert.
Run the fast foreign purchase test scenario and view the Inspector output:

You can see that a first online purchase comes in from Luxembourg. This first purchase causes 2 rules to fire: bronze customer status, as well as an alert that this is the customer’s first purchase from Luxembourg.

When the second online purchase from France is processed 15 seconds later the alert is emitted because the customer lives in the UK and has made 2 consecutive purchases from non-UK countries within a 2 hour time window.

Understanding variable bindings and constraints is the key to creating incredibly powerful patterns used to detect situations. Don’t forget to create a good base of test scenarios for your patterns, testing for: true positives, false positives, true negatives, false negatives. Don’t forget the basics and be careful when considering how your system will behave with respect to out of order events.