Requirements for rule engines
Capture and communication of complex business rules
Contrary to popular belief, the most difficult part of software development is not in the actual coding of systems but rather in the capture and translation of vague, incomplete, and often contradictory business requirements. Errors introduced early in the communication of requirements are the most costly system errors to correct, leading to patch development and increased maintenance costs. Nowhere is this more evident than in the attempt to capture and code complex business rules.
Business rules are defined in many ways, but in general can be best thought of as decision points in business processes. Business rules are described by "if-then-else"-type logic, where if a specific set of business conditions is met, then a particular action is performed; else (otherwise,) some other action may be performed. For example, consider a company offering a promotional rate for its services. The promotion is in effect for a limited time, for a specific set of products or services. The back-end processing systems must be capable of not only correctly applying the promotional rate to new customers but also of discontinuing the promotion at the appropriate time. These behaviors are driven by business rules.
The tricky part comes when these rules are coded into software programs. The majority of programming languages do not directly support the idea of independent rules matched and fired as a result of a set of conditions — this in spite of the large number of business processes that are rule-driven (see below).
Table 1. Examples of rule-driven business processes
|Financial decisions||Loan origination|
|Scheduling or routing||Package delivery|
|Product provisioning||Cellular service|
|Inventory management||Just-in-time supply chain|
|Fare calculation||Air, boat, train, or bus transport|
This article explores analytical techniques for the capture and modeling of complex business rules. As an example, I will use some hypothetical rules for water contamination testing. This subject area has well-defined regulations for determination of drinking-water contaminant concentrations as defined by the U.S. Environmental Protection Agency (EPA). For purposes of this article, I assume that the testing is performed by a government agency monitoring the quality of a local water supply. The intent is to show how you can combine simple rules into more complex structures expressing a robust business process.
Capture of business-rule requirements
Rules are different from use-case behaviors in that they describe business logic, rather than system behavior. In a typical use case, the flow of activity is driven by the interaction of a particular actor with the system toward the execution of a specific goal (for example, "managing a water-health sample"). By contrast, rules are only executed when the conditions for that rule are exactly matched as a result of other system processing (for example, applying payment against an outstanding balance). Consequently, the approach to capturing and validating business rules is quite different from use-case development.
Rules are typically structured as a well-defined pair of condition and
action statements. In general, it is a best practice to write rule
conditions as a set in which all conditions must be met for the rule to
fire. Although some rule engines allow for the
in a condition, in practice, such constructions lead to rules that are
difficult to understand and test. In these cases, it is better to create
multiple rules to cover each
or situation, rather than create
one complex rule with many possible matching scenarios. Therefore, in the
following examples, the default combination of conditions is always
and (see Table 2). Moreover, you should
create rules to execute as independently as possible, such that only one
rule is needed to perform a defined action. Doing so allows for
independent testing and validation of each rule separately, then these
rules can be combined into more complex behaviors.
Table 2. Rule categories
|Validation rules||Data verification, consistency checks|
|Calculation rules||Compute values based on input data|
|Decision rules||Selection of business process path|
|Generation rules||Creation of new data objects|
Individual rules are defined at an atomic level — that is, you can define a unique combination of conditions that will execute one and only one rule for that collection of triggers. A simple rule definition is shown in Table 3, where a water sample is evaluated for compliance with an established water standard. In this case, the action is simply to generate some form of notice, but you could perform a more complex action (such as the issuance of a citation) as the rule action. Notice that this rule definition can be validated and understood in isolation; there is no need to understand any rules that may have been executed prior to this one or any rules that follow.
In Table 3:
- The Rule name is: Evaluate Water Sample — Benzine
- The Rule group is: evaluate-water-sample
Table 3. Single-rule definition
|A non-voided water sample||Generate notice: Benzine limit exceeded||The maximum allowable amount of benzine in a sample is 0.005 mg/L.|
|Benzine concentration > 0.005 mg/L|
Sometimes, the presence of one condition modifies another. For example, consider a case in which new research shows that the permissible concentration of benzine (nominally at 0.005 mg/L) should be reduced in the presence of carbofuran (permissible limit of 0.04 mg/L) by 50 percent (new limit = 0.0025 mg/L). Such a rule could be captured as shown below.
In Table 4:
- The Rule name is: Evaluate Water Sample — Benzine (reduced standard)
- The Rule group is: evaluate-water-sample
Table 4. Single-rule definition, modified
|A non-voided water sample||Generate notice: Benzine limit exceeded||New research shows that the permissible level of benzine must be reduced by 50 percent in the presence of carbofuran.|
|Benzine concentration > 0.0025 mg/L||Generate notice: Benzine standard reduced (0.0025 mg/L)|
|Carbofuran concentration > 0 mg/L|
Notice that you now not only have multiple conditions but multiple actions.
You could have written these two rules using an
combination, but as noted above, that would complicate the understanding
and testing of each rule in isolation, which is why separating rules into
atomic units is a best practice.
Some rules are best represented by a decision table, which is just a way of summarizing many rules that share a set of conditions, with actions based on the changing values of those conditions. This situation is common in financial institutions, where decisions on granting a loan request are based on the requester's credit history, employment status, owned assets, etc.
In Table 5:
- The Rule group is: Determine Loan Eligibility
- The Description is: A loan officer will use this table to determine the eligibility of a loan requester.
Table 5. Tabular rule definition
|<10,000||>45,000||<10,000||Approve loan at 5-percent interest|
|<20,000, >10,001||>65,000||<10,000||Approve loan at 4.5-percent interest|
|<30,000, >20,001||>75,000||<10,000||Approve loan at 3-percent interest|
Each line shown represents a separate rule. Most of the commercial and open source rule engines support the ability to capture rules in a tabular rule form.
The final rule type is called a tree rule. Like the previous example, this rule-capture technique is simply a shorthand way of representing multiple related rules, but in this form, it allows for visualization of the full decision tree used to arrive at a particular conclusion. As shown in Figure 1, customers may be granted a discount based on the level of customer reward, geographic region, and the product suite they are receiving.
Figure 1. Decision tree
Rules may be closely related in that they match on the same input data or are performing a similar set of actions (validation rules, for example). It is often useful to group these rules to limit the rules considered for matching. Doing so is mostly for human observers, as rule-engine optimization algorithms are sufficiently powerful to optimize rule matching. However, there is a significant advantage in that if a particular set of data fails an early step in the processing, it is not considered for further processing with downstream rules, voiding the input. This exclusion simplifies the downstream rule conditions and reduces the likelihood of errors.
You use rule groups to organize sets of rules by well-defined categories to permit the creation of a rule flow. As shown in Figure 2, a rule flow may contain branch points and join points, which allows for very sophisticated management of rule execution. By using rule groups and flows, you can clearly model the overall rule engine behavior and validate the rules in isolation.
Figure 2. Rule group model
As rule sets grow more complex, it will become increasingly difficult for the business subject-matter experts (SMEs) to fully understand (and validate) all the interactions between rules and rule groups. Visual modeling of these interactions and dependencies provides a powerful mechanism for communication among the development team, testing team, and business stakeholders. The three model forms most useful in this context are a structure model of the rule groups, a dependency map among groups, and the execution flows that the rule engine itself performs.
As shown in Figure 2, I used the Unified Modeling Language (UML)
«stereotype» extension mechanism to enhance the UML activity
diagram to provide a visual model that captures this information. The
«rule-group» stereotype indicates a rule group, and
«rule» indicates a named rule. Figure 3 shows two ways to
«action» pairs for a
Figure 3. Rule UML model
This approach is useful when working directly with business users, as it presents the rules visually, rather than textually. Combined with the rule group-modeling diagram, these two model views can provide a full picture of the rules embodied in a business rule set.
Translating requirements into a rule engine
Requirement definitions need not be defined in the manner described above, but this approach readily leads to a simplified implementation that can be traced directly back to the specific requirements. As shown in Tables 6, 7, and 8, three example rules for Validate Water Sample, Evaluate Water Sample — Exposure Limit Exceeded, and Evaluate Water Sample — Fecal Coliforms are shown coded using the JBoss Rules (Drools) open source rule engine.
In Table 6:
- The Rule name is: Validate Water Sample
- The Rule group is: WaterSample
- The Priority (salience) is: 100
Table 6. Example rule definition to validate a water sample
|Void water sample||Generate notice: Voided sample advisory||The sample was marked as voided.|
In Table 7:
- The Rule name is: Evaluate Water Sample — Fecal coliforms
- The Rule group is: WaterSample
- The Priority (salience) is: 50
Table 7. Example rule definition for a Fecal coliforms water sample
|Valid water sample||Generate notice: Fecal coliform advisory||If any fecal coliform bacteria are found in the sample, a contamination notice is created.|
|Fecal coliform > 0|
In Table 8:
- The Rule name is: Evaluate Water Sample — Exposure Limit Exceeded
- The Rule group is: WaterSample
- The Priority (salience) is: 50
Table 8. Example rule definition for an Exposure Limit Exceeded water sample
|Valid water sample||Generate notice: Exposure limit exceeded||The sample exposure limit is included in the sample data collected.|
|Contaminant conc. > exposure limit|
Figure 4 shows the Drool implementation of these three rules.
Figure 4. Drools implementation of three water-sampling rules
Depending on the rule-engine implementation and the interaction between rules, there is often a need to prioritize the execution of rules. There are two techniques for doing so, the first being the previously mentioned rule groups and rule-execution flows. The second technique is at the individual rule level, where you can give a particular rule within a set priority over the others. (In the Drools engine, this is called the salience level.)
There are times, such as when validation rules are to be applied prior to executing other processing rules, when one rule must execute before another, even though both rules are valid selections for firing (that is, the rule conditions have been matched). In these cases, the rules may have overlapping condition sets, or one should not be executed unless the other is performed first.
In the example above, even though the two rules are in the same rule group (WaterSample), the evaluate rule will execute after the validate rule based on the provided salience level. Assuming that the sample fails the validation, the sample data can be retracted from the working memory so it no longer is available for matching the following rules.
Testing and validation
When you have implemented the business rules into the rule engine, the final step is to ensure that the coded rules match against the requirements. Many rule engines provide testing and evaluation tools, but sometimes it is necessary to create specialized tools. As shown in Figure 5, a tool written to supplement the testing tools provided with the Drools engine allows the testing team to select validation data on the fly and immediately review both the results and the executed rules.
Figure 5. Rule evaluation and testing
As shown, rules you define, capture, and implement as atomic units can be far more readily tested for correctness. Using automated tools further allows for repeatable testing to ensure that as new rules are added to the collection, the older rules are not broken. Note also that in this example, the tool includes the ability to maintain the working state of the rule engine, which essentially allows for stateful on-the-fly testing — useful for testing situations in which data must be present in the rule engine from previous processing to test the execution of a set of rules.
Encoding business rules in software systems is a necessary and challenging task at the best of times. Presentation of ambiguous, incomplete, or misleading requirements simply compounds the problem for accurate development and leads to expensive errors. The combination of visual and textual descriptions of business rules allows for a more accurate and effective way to capture business rules, especially for implementation using a rule-driven engine. Using the techniques described here, you can better equip business analysts with a powerful analytical framework for communication of complex business-processing rules.
- Read "Extending the power of the WebSphere Process Server business rules component" to get an understanding of the IBM® WebSphere® business-rules engine.
- Learn more about UML and how to extend it from the source from the Object Management Group.
- "A Practical Method of Developing Natural Language Rule Statements" is a series of articles on natural-language business rules.
- The JBoss Drools rules engine is a stable and good alternative to commercial rule-engine products.
- Follow developerWorks on Twitter.