Daniel Selman 2700022VQ3 Visits (3121)
Within the ILOG BRMS team we often talk about "top-down" rules development versus "bottom-up" rules development. These terms are pretty vague, and when I use them outside ILOG they seem even more vague! I thought it would be useful to blog on this topic, stimulate some discussion and perhaps convergence of concepts and ideas.
Wikipedia has an interesting article that brings together a lot of thoughts on top-down vs. bottom-up from many different fields: engineering, politics, ecology, architecture etc. These concepts are probably so universal because they correspond to basic human problem solving techniques. Faced with a complex object many children will try to take it to pieces, and then try to put it back together. "Taking things to pieces" is decomposition; starting from a complex whole and determining its constituent parts, while "putting it back together" is assembly. They require different skills, particularly as we move from the world of physical objects to abstract concepts.
I'd argue that effective architecture requires rapid back and forth switching of context from designing the whole to designing the parts. If no one can see most of the forest and most of the trees, you usually end up with a system that either solves the wrong problem, or that breaks-down when applied in practice.
If you apply decomposition to the BRMS application design process you end up with something like the Zachman Framework for Enterprise Architecture:
Note that the business rules cross-cut all the questions you have about your business: "what, how, where, who, when, why", and are not merely relegated to describing the motivation column. While the CEO is defining the business strategy at the "top" a DBA might be proposing changes to database table structures at the "bottom". Sometimes along the way these two proposals collide -- leading to a development challenge that I call "meet in the middle" development.
On Zachman's diagram the typical "business" concerns are in the upper slices, while the more technical IT-centric concerns are in the lower slices. The goal of BRMS technology is to bridge these two constituencies -- so it is no surprise that we are frequently asked how to organize development: top-down or bottom-up. The reality is that in many cases we are confronted with an iterative "meet in the middle" scenario.
The majority of customers use a bottom-up modeling approach with JRules and Rules for .NET. Historically this is been one of the technical strong-suits of the products: allowing easy generation of a Business Object Model (BOM) from existing Java or C# classes, a rule language that allows access to the fields and methods within the BOM, and very high performance/low overhead execution against instances of the XOM. In addition many customers are not performing new (green-field) development, but already have a rich domain model and existing services that they would like to access from business rules.
Once the BOM has been generated from the Java, C# or XML Schema Executable Object Model (XOM) the BOM may be flattened and refined to make it more suitable for rule authoring. Business analysts will edit the Vocabulary for the BOM to ensure that suitable business terms are used for the methods and attributes within the BOM.
For new applications customers may work their way down each slice of the Zachman framework, fleshing out the detail of the models required, based on the business requirements from the row above. When they get to the level of the conceptual business model they will define a BOM and a Vocabulary that verbalizes the BOM for a target language. The BOM/Vocabulary may be expressed in JRules terms or may be a higher level (vendor neutral) artifact such as a UML model plus a textual glossary or an OWL ontology etc. In most cases it is translatable into JRules BOM/VOC artifacts however.
Some algorithms and products may even help generate models/ontologies from source text, TextVis, TextOntoEx, or BioLexicon for example. You can imagine processing an existing employee handbook to generate a seed model used to bootstrap rule capture and development.
Because the BOM has not been imported from a XOM any authored business rules are parseable, but not yet directly executable. To make the rules executable a dynamic loosely-typed XOM could be used (using Maps/Lists to define "slots", accessed using B2X) or a static Java/C# XOM could be generated from the BOM -- either as source or byte code.
Iterative Rule Authoring
Once an initial seed BOM/VOC is in-place rule authoring may proceed, using either a bottom-up or a top-down process. In some cases business rules may need to be "mined" from a legacy system (bottom-up), in other cases knowledge engineering may take place, capturing human knowledge from subject matter experts using interviews and reviews of process documents and best practices (top-down). Again, some text, database or source code mining tools may automatically extract a highly variable fraction of the business rules. The quality of the rules automatically extracted from technical artifacts is also highly variable, and they often require substantial analysis before they can be efficiently used with the BRMS.
It is quite likely that the process of extracting legacy business rules or knowledge engineering will impact the terms in the vocabulary as well as the BOM. Iterative refinement is then required, which may require refactoring (updating) existing rules to ensure they work with the latest vocabulary.
Meet in the Middle
As new rules are captured, they may introduce new concepts into the vocabulary or modify existing concepts. As developers modify existing XOM domain classes, they may add new members (attributes or methods). These changes can then be imported into the BOM by updating the relevant classes. These two change vectors must be reconciled using a meet in the middle development paradigm. Old BOM members should be deprecated (ensuring that a warning is generated if new rules are authored against them), while new vocabulary concepts can be introduced by editing the vocabulary. Incompatible changes to the XOM can be compensated for by updating or introducing B2X mapping code. Changes to the vocabulary concepts used by rules will trigger automatic refactoring of the rules when possible.
On paper this sounds relatively straight-forward, however in practice, "meet in the middle" is a real challenge. It requires strong change management processes, good rapport and coordination between the technical and business teams and a solid understanding of the quality and testing implications of different types of changes. While JRules provides substantial tooling to help with "meet in the middle" challenges: B2X, BOM life-cycle management, BOM Update and refactoring of rules when the vocabulary is edited, we are aware that there is still more to do in this area. For example, in JRules 7 we have re-implemented the BOM Update backend and GUI to be more responsive, usable and powerful -- making it easier to synchronize the changes in the XOM with evolution of the BOM.
I'd be very interested to hear about your challenges in this area. Please use the comments area below.