In this rapidly changing business climate the time it takes to react can be the difference between success and failure. A solution constructed of business process management (BPM) and business rules systems, with access to the latest business intelligence can enable this kind of responsiveness. This is the first of a series of articles that will look at several scenarios for creating flexible, dynamic, and responsive solutions through the integration of BPM, business rules, and business intelligence.
Becoming an on demand business
Businesses today are facing many pressures--pressure to compete in a fast-paced ever-changing climate, pressure to constantly reduce costs to remain competitive. Becoming "on demand" means preparing your organization for success in this difficult climate.
There are several dimensions to becoming an on demand business.
- First, there is creating a responsive environment that can quickly react to any change in the marketplace whether it is internal or external, planned or unforeseen, a threat or an opportunity.
- Second, the organization should have variable cost structures to manage costs in proportion to growth of the organization or increase in demand.
- Third, the organization needs to be focused on what is profitable and core to the enterprise's success.
- Finally, the organization needs to have a resilient robust infrastructure that is available around the world and around the clock.
But the question is, how does one create a responsive environment? To answer that question, let's take a look at International Foods Market (IFM), a fictitious company, and their challenges. As a typical large retailer they offer thousands of products from hundreds of suppliers. To succeed they must carefully manage inventory, markups, labor costs, and other dimensions of their business. They need to respond quickly to any challenge their competitors bring, improve their customer service, get new products into the stores faster, and rapidly adopt new governmental and industry standards.
Christina White, the CIO, has a major challenge on her hands. How does she create an IT environment that can be more responsive to these challenges? Look at the typical way her organization has handled changes in the past. A requirement would come down from corporate, which Christina would assign to a project team, which would size it and schedule the development of changes to legacy applications. These legacy applications are large monolithic programs, and so often new requirements take anywhere from months to years to implement. This is illustrated in Figure 1.
Figure 1 - Monolithic application
At a recent CIO conference, Christina heard about the benefits of business process management technology and how the separation of the business process outside of the application logic can create a flexible responsive structure that can be easily modified or extended as the company's business processes change. She decides to have one of her architects, Greg Thomas, investigate this. Greg in his digging discovers that not only is there tremendous benefit from moving the business process outside of the application logic into a BPM engine, but business rules engines offer the ability to move the company's policies into an easily controlled component as well. Christina likes having easy access to the corporate policies and critical business processes so they can easily modify them to respond to new requirements.
Greg also mentions that the data warehouse deployment, which was recently completed as part of their corporate CRM initiative, is an excellent source of business intelligence to use within the context of the business processes and rules environments. They have seen requirements in the past for better targeting of promotions and more flexible pricing solutions that could benefit from access to this BI data. They decide to integrate the data warehouse into this new environment.
As you can see in Figure 2, applications, new or existing, provide the instantiation of the business logic, but flexibility is brought into the equation by implementing the business process using a Business Process Manager like WebSphereamp;reg; Business Integration. Likewise business policy is implemented in a Business Rules Engine. This new architecture brings several benefits to the IT environment.
- First, by consciously separating out policy and process from the logic, the organization is forced to clearly think about what their processes and policies are and what they should be.
- Second, the rich tooling of these technologies makes it easy to rapidly and incrementally change the process or policy (remember the "Variable" on demand objective).
Figure 2 - Flexible BPM application architecture
The spreadsheet was an instrument of change. When the first spreadsheet, Lotus 123, came out, most people who were analyzing their data were doing it by hand on calculators. Remember reverse polish logic? Only so much analysis could be done with a calculator, but when the spreadsheet arrived all of a sudden each individual was free to dream up all kinds of calculations they could do. This change created whole new ways of looking at data.
Business process management is the same way. It is not just a technology for running business processes; it is a vehicle for change - for process improvement. BPM technologies allow organizations to systematically understand, analyze, improve, and implement their business processes. BPM is a revolutionary approach to improving business operations through information technology.
There are multiple benefits of BPM:
- It creates a common process base independent of any application.
- It allows people to organize their business processes and observe them visually.
- It facilitates the formalization of business processes that previously had been undocumented and sometimes ad hoc.
- It provides a vehicle for instituting conformance of behavior where conformance is beneficial to the organization and flexibility where conformance is not necessary.
- It allows people to model, simulate, and analyze business processes so they can create a business process that maximizes their objectives.
- It serves as a way to speed up the implementation of process changes. As Fingar and Smith put it in their book, BPM the Third Wave - "BPM doesn't speed up applications development; it eliminates the need for it."(1)
- It allows an organization to focus on processes that are "Core" and even to consider offering those processes as services to trading partners.
Figure 3 - Business Process Management
A business process is an activity in an organization that has a finite beginning, a set of intermediate activities, and a final outcome. Processes have interested parties, called stakeholders, who benefit or participate in the business process. Not all business operations are business processes. For instance, hiring a new employee is a business process, whereas that employee clocking in to begin work is not.
There are some characteristics that quite often exist in any business process. Business processes are large, complex, dynamic, distributed across multiple parts of the organization, long-running in duration, and either human interactive or automated. Business processes serve to bridge between IT and the business environment, requiring that the tooling and technology be useful to both business and technical people.
What is WebSphere Business Integration?
WebSphere Business Integration offers a set of functional components that provide business integration messaging, integration brokers, and business process management solutions. There are two business process management technologies available in WBI. WebSphere InterChange Server (formerly CrossWorlds) provides process automation that manages multiple discrete business applications as one. WebSphere MQ Workflow supports long-running business process workflows as they interact with systems and people.
Complimenting WebSphere InterChange Server and WebSphere MQ Workflow is a set of tooling for modeling and monitoring business processes. WebSphere Business Integration Modeler provides the tools to design, test, and communicate complex business processes. WebSphere Business Integration Monitor displays real-time information from a variety of environments to allow decisive business performance management and optimization.
Let's return to International Foods Market. As part of their first BPM project, a team of technical and business people begins to record their organization's current business processes using the WBI/Modeler tool. The tool helps them capture the step-by-step details of the "AS IS" process. They discover some interesting things which they never about their organization:
- The functionality of one of their legacy systems does not really fit well with the actual way that the employees found for getting their job done, so the employees are doing several of the steps in the process manually.
- The steps for determining the price of products is imbedded in a packaged application they bought long ago, and no one really knows how to change the rules for determining price mark up. The person who initially developed the system specifications has long since retired.
Christina decides that the pricing problem would be a good one to tackle in one of their first implementations. This is too important an element of IFM's business to leave locked inside an application that no one can get at.
In the traditional application structure, the business rules are buried in with the business logic inside the application. This application imbedding of business rules requires changes to the applications code every time the rules need to be changed. This limits the ease of customization and extension. However, moving the business rules outside of the application empowers the business analyst to describe the processing rules independent of the application logic.
This enables adaptability to respond to changes in:
- Corporate policy
- Government regulations
- Customer status
- Contract terms amp; conditions
- New business models
Business rules are meant to encapsulate chunks of business logic that are subject to change. One key to succeeding with rules engines is to carefully identify those pieces of logic that are "highly volatile." These chunks can be exposed as rules and thus easily modified though the business rules engine tooling. Rules engine facilitate late binding where the specific linkage from business logic to business rules is not defined at implementation time or even deployment time, but is a run-time decision. This creates maximum flexibility.
In the same way that business rules enable the separation of corporate policy from the application, a business process manager facilitates the separation of corporate business processes from the application logic. Using a component-based model of the application constructs together with a business process manager and business rules engine provides maximum flexibility and adaptability.
Rules engine systems are available from some leading independent software vendors including Versata, ilog, and Blaze, and also in the BRBeans technology in WebSphere. While each is different in features and functions, they all offer some of the same basic functional benefits:
- Externalization of business policy or rules at points of variability where business rule call-outs are inserted into the application code or business process flow
- A language for expressing rules in a straightforward way
- An execution environment for running the rules
- Tooling to allow users to create and modify business rules.
Figure 4. Business rules engine
It is worth noting that there are different types of rules engines and it is extremely important to understand the purpose of a tool before deciding how to apply it. When you're surveying across the business rules domain, you'll note that the tools may be classified into the following categories:
- Simple business rules - Business rules that are expressed through a simplified straightforward vocabulary and are invoked through points of variability in applications or business processes. Good examples of this type of rules engine are ilog, Blaze, and IBM's BRBeans.
- Artificial intelligence rules - Rules that govern the behavior of algorithm in AI and Data Mining products. One example of this type of rules Engines is the DB2amp;reg; Intelligent Mineramp;#153; product.
- Event correlation rules - Rules that are used in the correlation of events in order to aggregate a set of isolated events into an aggregated meaningful situation. One good example of this type of rules engines is the Tivoliamp;reg; Event Console systems management product.
- Data-centric rules - These are rules that constrain the retrieval or updating of data. These constraints control how the data is transformed and who can access it, and ensure that data integrity is preserved through syntactical, semantic, and contextual enforcement. A good example of this type of rules engine is Versata.
- Transformation amp; validation rules - These are rules that define the modification of data either in application integration or information integration scenarios. These rules define how data is modified, cleansed, or validated. Products that provide these types of rules include WebSphere Business Integration and DB2 Warehouse Manager.
Business intelligence, as its name implies, describes is a set of technologies whose purpose is to create intelligent knowledge from the chaos of corporate information. The volume of information accumulated by an average enterprise can extend into the terabyte range in a relative short period of time. Without an understanding of the insights that the data wants to tell them, organizations will not win in the marketplace. This can be seen from a quote from a recently published book:
"Knowledge drives business competitiveness. Winning in the marketplace requires business knowledge-knowledge about your customers, your products, and your processes. Only then can your business be tuned for excellence in the marketplace."(
2)
Before delving into how business intelligence relates to business process management, let's take a little time to discuss what a business intelligence system looks like. The cornerstone of a business intelligence solution is the data warehouse, a repository of the historical record of every meaningful event and record that has occurred in the past. Surrounding the data warehouse are components that assist in the feeding, analyzing, and reporting on the data in the warehouse. Let's take a look at these components:
Data sources - The data warehouse ecosystem begins with operational systems and operational databases that run the enterprise. The data from the various systems provide the fragments of information that must be pieced together to create a complete historical record of business operations. These data sources can include legacy systems, packaged software applications, ERP systems, and relational or even non-relational databases.
Extract, transformation, and load (ETL) -- All organizations have a great diversity of business applications that vary widely in syntax and semantics of their data. When data from these data sources are brought together for loading into the data warehouse, their inconsistencies need to be resolved. This process of getting the data from disparate systems into the data warehouse is called the ETL process. In this process data is extracted from each of the data sources on some scheduled interval, moved to the data warehouse environment, transformed into an agreed-upon schema, and loaded into the data warehouse. Often a cleansing process is needed for data that has been corrupted by user input or application errors.
Staging tables -- These serve as intermediate tables between the operational environment and the data warehouse. They allow the proper sequencing of updates to the data warehouse by allowing time independence between the ETL process and the coordinated updates of the full historical record in the data warehouse.
Data warehouse -- The business data warehouse (BDW) is the central repository of historical information. It is the complete historical record of the business operations. If an event or data value is not captured in the warehouse, then it may be lost for all time, preventing incorporation of that piece of data in some future analysis. This is why it is so important that the data warehouse be designed to house the complete historical record of business operations. While most data warehouses are built on top of RDBMS, their structure is slightly different from traditional business operations relational databases. The data warehouse has a star schema that serves to enable the ad hoc "what if" querying that is done while analyzing historical data. The star-schema consists of a central fact table containing the business measures surrounded by dimensions that provide this flexibility. Typical dimensions include product, customer, region, organization, and of course time.
Figure 5 - The Data Warehouse Ecosystem
Materialized query tables -- The data in the data warehouse is atomic and too low level for human interpretation. If a particular retail store sells 100,000 items a day then that would be 100,000 new entries in the data warehouse each night. To be truly meaningful this data needs to be summarized at higher levels such as sales per hour or sales per day. DB2 offers a facility to automatically create these summarized data values through what are called materialized query tables (You may know them by their previous name - automatic summary tables). These tables provide an important bridge between the atomic historical records and the slice, dice, and drill structure of the multi-dimensional cubes.
Multi-dimensional cube -- Business analysts have always had a thirst for knowledge. In the old days of database reporting tools like QMF®, they would keep the people responsible for report creation busy with constant requests for changes in these reports. To alleviate this requirement, a new data structure was created that allows the business analyst to dynamically change the report in real time by slicing, dicing, and drilling through interactive reporting applications. The structure that enables it is a multi-dimensional cube. This cube allows easy navigation across the various dimensions of the cube. Specific views or data marts of the full data warehouse can be created by creating a new multi-dimensional cube. DB2 OLAP Server and DB2 Cube Views are products from IBM that can be used for building multi-dimensional cubes.
Online Analytical Processing (OLAP) - The reporting environment on top of the multi-dimensional cube is done through a standard reporting interface called OLAP, which allows various OLAP reporting engines to work with different multi-dimensional data warehouses. The OLAP interface enables this slice, dice, and drill behavior. OLAP reporting engines enable the creation of printed reports or dynamic interactive reports that users can access through desktop or Internet reporting applications. They offer rich sets of graphical components. For instance, alphablox, an OLAP reporting engine, offers over 300 different chart types.
So what usefulness does business intelligence bring to business process management? Obviously there are tons of benefits. As we stated before, business decisions implemented in business processes need to be based on the best insights into corporate intelligence about its operations, competition, and customers. Business intelligence exists not just for creating reports, but should be actionable, empowering business process advances.
Does the "I" in IT merely imply the technology opens the firehouse of data towards the user, or is there some higher degree of intelligence that it seeks to provide? Fingar and Smith in BPM the Third Wave, argue that information should empower process change:
"The reality is that to this day, computers are record-keeping machines, not management machines. They can take in chew up and spit out trillions of bytes of data, but where is the management insight, the actionable information needed in context, in real time at all levels of automated and human decision-making?"(
3)
The information ecosystem in an organization should flow from business operations through the data warehouse to be transformed into actionable knowledge that flows back to the operational environment to realize improved business operations. This is often called closed-loop business intelligence because of the full cycle of process improvement that is realized by integrating the results of analytical analysis back into business operations.
Closed-loop business intelligence at IFM
Returning to International Foods Market, remember that dynamic pricing is one of the scenarios they are tackling. Christina and Greg meet with some of the key business people who understand pricing the best. Together they begin to define the business process for setting and changing prices as well as the business rules to determine what the price should be. In creating the business rules they see that they need to include profit as one of the elements in the formula.
As they begin to think about this, it is not as simple as they first thought. Profit for any one product fluctuates quite a lot because of variability in the costs of buying the product from the supplier (cost of goods sold), the shipping costs, the warehouse or shelf-space to hold the product, and the labor cost to stock the product in the stores. Greg identifies several alternatives for solving this problem:
- They could just have a fixed value for profit for all products.
- They could create a static profit table with the profit for each product based on the value in a current business intelligence report.
- They could get real-time access to the profit value in the data warehouse through some kind of linkage from the BPM environment to the data warehouse.
Both Christina and Greg agree that real-time access to the latest data warehouse calculations for profit would be ideal. This is a great example of being an on demand business allowing IFM to be extremely responsive to changes in sales, inventory, and supply chain. We will return in future articles in this series to how this integration is accomplished.
We can't complete a discussion of the integration of business intelligence and business process management with touching on one of the hottest areas of technology that is all about BI+BPM. It is business activity monitoring.
The term business activity monitoring (BAM) was coined by Gartner Group a couple years back to describe some of the emerging capabilities that were bringing several key technologies together to radically change the landscape of business systems. BAM emphasizes several key concepts of business improvement that are capturing the hearts of CxOs across the industry:
- Business process management revolutionizes how business operations can be managed.
- Business processes are driven by actionable intelligence.
- Business processes can be improved by analyzing key measures captured during the execution of the business process.
- Businesses do not want to simply continue to spit out thousands of pages of business intelligence reports. They want the business intelligence system to report on key performance indicators (KPI) that reveal the critical calculations of operational success. This has lead to a new discipline called enterprise performance management that focuses on computing, reporting, analyzing, and controlling the business based on these KPIs.
- Zero latency environments need to exist that will in real time transform business events into actionable knowledge that can be leveraged by business processes and operations.
BAM can be thought of in terms of the "Five R's" of business activity monitoring. These are:
- Recognition,
- Response
- Resolution
- Review to function
- Delivering ROI
In keeping with the idea of actionable business intelligence, a BAM solution should be able to recognize what is going in the environment, interpret the meaning of this information, and decide what to do about it. Events are occurring around-the-clock. It has often been said that most people can handle dealing with at most 7 things at one time. For many managers in this face-paced world, this limit can be exceeded by 8:15 each morning. This is why automatic detection of significant business events is critical.
Figure 6 - Business activity monitoring
Recognition of the significant events requires sophisticated analytics and business rules systems that can interpret the events in the larger context of other events and the past history of business operations. One bee sting is annoying but not necessarily harmful unless you are allergic. However when you receive a second or third bee sting, then problems are most definitely on their way. In the same way any one business event by itself may not be significant, but together with other events it may become very significant.
BAM is a convergence of business intelligence and business process management as illustrated from this quote from a recent article in IntelligentEnterprise Magazine:(4)
"Is BAM about process automation and workflow management, or is it about Business Intelligence and performance management? You could argue that BAM is the next level in Business Process Management or a key component in any corporate performance management system. In reality, it's both. It can help to optimize business processes and as a result improve your understanding of both operational and strategic performance."
Now let's introduce three scenarios at IFM that we will be discussing in future articles. These scenarios demonstrate the value of bringing together business process management, business rules engine, and business intelligence to create a responsive, dynamic on demand environment.
Scenario one - dynamic pricing
Christina has a requirement to replace IFM's legacy pricing application with a new flexible pricing model that takes into account key supply chain, sales, and customer information. The business people want the ability to change pricing rules based on conditions in the marketplace. There are different pricing models they want to use based on characteristics of the type of product, customer responsiveness, and competitive pressures. They cannot determine all of the factors they want to use in the pricing activity or the pricing rules, but they want the flexibility to be able to add new factors or easily change pricing rules.
Some of the conditions that the business people want to be able to react to include:
- Changes in shipping or costs of goods sold
- Radical changes in inventory levels
- Undercut pressures from a competitor
- Seasonal effects on product demand
To create this kind of flexibility that the business people desire, Greg realizes that business process management is the ideal solution. As new requirements arise for the pricing process they can quickly change the process flow using WBI process design tools. The requirement to apply different pricing equations could be met by exploiting a business rules engine that would be integrated into the pricing business process.
Figure 7 - Dynamic pricing
This scenario is an example of the proactive business model where IFM is focused on correctly pricing their products based on marketplace conditions before a problem arises. It allows IFM to anticipate fluctuations in price responsiveness and set the price accordingly.
In the next article in this series we will go into the details of how business processes in WBI Interchange Server can be integrated with a business rules engine in support of this first scenario.
Scenario Two - Integrating business intelligence into BPM
In scenario one, one of the requirements for the pricing business process was to include access to historical and analytical information in the business process to provide a current profit value indicating IFM's realized profit that it is currently being reported in the data warehouse. This brings a real-time nature to the business process allowing IFM to quickly react to fluctuations in profit for a particular product or product category.
The integration of business intelligence into a business process has some difficulties. First, there is a broad range of calculations for profit that could be created. Do they want to use the average product profit by region for the last month, average product category profit at the corporate level for the last quarter, or average product profit at the store level for the last month? The integration solution needs to address this requirement for flexibility.
The second issue is dealing with the computed values in the multi-dimensional cube that is part of the requirement. Flat relational tables are easily accessed in WBI Interchange Server through RDBMS connectors, but what about the multi-dimensional data residing in the data warehouse? In our third article in this series we will discuss techniques for providing access to data warehouse information in the operational business process environment.
Scenario three - sense amp; respond
Christina is pleased with the management feedback on the new proactive pricing business process. Recently she was called into the senior management meeting to be complimented on the impact that this new business process has made to corporate sales and profit. However the enthusiasm around this solution has highlighted additional challenges for the IT team. The management team has noticed there are conditions that are occurring in the stores and in the supply chain that are not being detected. If they could be detected then this pricing business process could be initiated to react to the problems.
Figure 8 - Sense and respond monitoring
This reactive process would monitor various business measures that could indicate there is a problem with the sales of a particular product. With well over a thousand stores that sell an assortment of over 10,000 different products, it is very difficult for IFM corporate personnel to monitor all possible deviations in sales and inventory levels of a particular product. These conditions could be symptoms of problems including:
- Possible loss of sales to a competitor
- Possible supply chain problem (product not shipped or too much product shipped)
- Possible diminished demand for a particular product
IFM would like to begin to monitor information in the data warehouse to detect these conditions and then to either notify a particular employee to take action or to initiate a business process that could change the price, start a promotion, or place an order on the supply chain. In our fourth article in the series, we will talk about how to create monitors on top of the data warehouse to detect deviations in product sales and inventory.
Stay tuned for the subsequent articles in this series where we'll dive into the details of how BPM, business rules, and business intelligence systems can be integrated within the context of these business scenarios at International Foods Market.
1 Howard Smith and Peter Fingar, Business Process Management the Third Wave, Meghan-Kiffer Press, 2003.
2 John Medicke and Rob Cutlip, Industry Solutions with DB2, Prentice Hall PTR, 2003.
3 Howard Smith and Peter Fingar, Business Process Management the Third Wave, Meghan-Kiffer Press, 2003.
4 Stewart McKie, "The Big BAM", IntelligentEnterprise Magazine, July, 2003.
John Medicke is the chief architect of the On Demand Solution Center in Research Triangle Park, NC. He has worked in industry solution development for last seven years across various industries including financial services, retail, health care, industrial, and government. He is the author of the book Integrated Solutions with DB2 as well as multiple articles in various journals. You can contact John at medicke@us.ibm.com.
Feng-Wei Chen works for IBM at its Research Triangle Park, North Carolina location. She is a software developer in the Software Group On-Demand Solution Center. She has participated in several solution integration projects. She has been involved in the design and architecture of solutions related to database systems and business intelligence for three years. You can contact Feng-Wei at chenf@us.ibm.com.
Margie Mago works for IBM at its Research Triangle Park, North Carolina location. She is a developer and solution architect in the Software Group On-Demand Solution Center. She has participated in several retail sector solution integration projects since joining the group. You can contact Margie at mmago@us.ibm.com.
Comments (Undergoing maintenance)
Table of contents
Next steps from IBM
My developerWorks community
Tags
Use the slider bar to see more or fewer tags.
Popular tags shows the top tags for this particular content zone (for example, Java technology, Linux, WebSphere).
My tags shows your tags for this particular content zone (for example, Java technology, Linux, WebSphere).
Popular article tags |
My article tagsSkip to tags list
Popular article tags |
My article tags
