A basic DSI rule takes the form:
<when event occurs, where condition> <if event condition and bound entity state> <then set entity state; emit event;
You can access the full reference manual for the rule language here. In this article we will look at the considerations for writing basic rules that fire when an event occurs. In subsequent articles we will look at intermediate and advanced rule writing topics.
Here is an example of a simple rule from the InsightsStarter solution:
when a purchase occurs, called LAST if the total amount of all order items in the order items of LAST is more than 100 and the customer status of 'the customer' is NONE then set the customer status of 'the customer' to BRONZE; emit a new customer notification where the customer is 'the customer' , the message is "Congratulations you are a bronze customer.";
The rule is triggered by the arrival of a purchase event, and the purchase event contains a set of order items within the event. The total amount of all order items is calculated and if the total is more than 100, and the customer status of ‘the customer’ is NONE, then the customer status is set of BRONZE and a customer notification event is emitted.
All stateful DSI Agents have a single bound entity, declared in the Agent Descriptor (ADSC) file for the agent. In this case the Rule Agent descriptor declares that the bound entity is ‘the customer’ and specifies how to retrieve the customer reference from the incoming purchase event.
'demo_rule_agent' is an agent related to a customer, processing events : - purchase, where this customer comes from the customer of this purchase
An Agent Descriptor that specifies that the bound entity for the agent is a customer entity, and subscribes the agent to purchase events.
When a purchase event is received the purchase event is routed to the container in the grid that hosts the primary copy of the customer that performed the purchase. The event history for the customer is then retrieved, and the agent runs. The agent has read-only access to the event received, read-only access to the event history, read-write access to the bound entity and read-only access to entities that are not the bound entity. DSI enforced read-only access is intended to help achieve high performance and linear scalability.
To get good performance the size of the bound entity should be as small as possible, as the bound entity must be read, written and replicated across the data grid for high-availability. In particular you should be careful that you do not “leak” data into the bound entity by adding information to it that never gets removed.
Workload Distribution and Hot Entities
The bound entity is used to partition and distribute work across the compute grid, with agents running on the node in the grid that hosts the primary copy of the bound entity. Moving the compute to data improves performance. One must be aware however that if the incoming events are not evenly distributed across the entities then the compute workload will not be evenly distributed across the grid. I.e. if 90% of the purchase events in the example above are for customer DAN, then we’d expect to see 90% of the events processed on the machine that hosts the primary copy of customer DAN. Entity DAN has become what we call a Hot Entity.
Because a given agent with the customer bound entity needs a consistent view of the events it has received, read-write access to a bound entity is serialized. We’d expect to see contention and queuing for access to DAN. The runtime performs sophisticated thread management to try to minimize the impact of Hot Entities, however they will impact throughput, machine utilization and overall performance of the solution. You should try to avoid Hot Entities where possible, for example by a finer partitioning of your incoming events. Where they cannot be avoided they must be taken into account when sizing the hardware required to run the solution.
The biggest influence on the performance of a rule agent is the size of its event history, in number of events. Looking for patterns in an event history of 1,000 events is much more expensive than looking for patterns in an event history of 20 events. The runtime also has to do much more work to make the event history highly-available so that machine failures can be handled with no downtime.
If you are merely processing the current event using the type of rule shown above you do not need to retain any event history. You should therefore set the time horizon for the event history to 0 days to ensure events are never retained in the event history. You can do this globally for a solution by setting the maxHorizon property, for a given agent, or per event type in an agent descriptor.
'demo_rule_agent' is an agent related to a customer, processing events : - purchase, where this customer comes from the customer of this purchase, with a horizon of 0 days
In some cases you do not need to look for complex patterns in event history, but you do want to count the historical occurrence of events that meet some criteria. To do so you can use Shared Aggregates (new in v8.8).
Shared Aggregates are defined in the Business Model Definition. For example:
the total transaction amount of a customer is aggregated from transactions , where this customer comes from the customer of each transaction as the total amount of all transactions available for 365 days with a resolution of 6 hours.
The state for shared aggregates is maintained outside of the event history; they can therefore be used by rule agents that are processing events, but specify that the events should not be retained. You can read more about shared aggregates here. You can use a shared aggregate value in a rule condition just as you’d use a numeric entity attribute – the only difference is that shared aggregate values are read-only as they are automatically updated on receipt of an event.
Controlling Rule Firing
Thought should be given for how to potentially stop these stateless rules for firing on every event. The simplest mechanism, as shown in the Bronze rule above, is to simply set a field in the bound entity to supress rule firing. When the status of the customer is NONE the rule will fire, and when it fires it sets the status of the customer to BRONZE, which will prevent the rule for refiring given another purchase event.
Subsequent articles will discuss more advanced mechanisms, based on time conditions or using event history.
When writing rules you should be aware of potential recursion issues. The simplest form of recursion is if an agent emits an event type to which it is also subcribed. Harder to spot cases can involve an agent emitting an event type A when the agent is subcribed to a super-type of event type A. You can also create cycles, where an agent X emits A, and an agent Y emits B, and agent X is subscribed to event type B or a super-type of B.
The runtime will spot recursion and will throw an exception to break the recursive loop. You can control how much recursion is permitted/expected in your solution by setting the maxRecursionCount global property.