Overview
The Information Aggregation pattern, also known as User to Data, facilitates user access to and manipulation of data. Information Aggregation functionality is often also called one of the following names:
- Management information systems (MIS)
- Decision support systems (DSS)
- Data warehousing
- Business intelligence (BI)
Designing applications that automate the Information Aggregation business pattern can be challenging for many reasons. User requirements in this area of e-business tend to be vague and constantly changing. Prioritization of needs across a company is challenging. Substantial infrastructure is often necessary. And often, several business intelligence applications must be built simultaneously. Some of these may have common data needs while others may have conflicting needs.
To overcome challenges like these, best practice suggests that two separate steps be used to aggregate and distill structured and unstructured data. To implement the Information Aggregation business pattern you may need to pre-populate either derived data stores or indices before you can execute the User Information Access pattern. This separation allows for greater flexibility in changing either the population function or the information access function without impacting the other. The separation further promotes component reuse as well, one of the main goals of the Patterns for e-business model.
The Population and User Information Access functionalities of the Information Aggregation business pattern are first described in general terms here.
Population
Population involves designing and creating applications to extract, cleanse, restructure, and move data into or between appropriate data stores. The population step is needed if the required data does not already exist in the appropriate data store, or if the data is not in an optimal form to satisfy the user's needs.
Consider a large grocery store chain. Location managers of these stores would like to receive a daily report summarizing perishable items that must be placed on sale in order to clear the inventory before these items expire. Such a Decision Support System (DSS) would need to distill information from a vast inventory data store. Inventory data is most likely not optimally structured to run such reports. A Population application must be developed that extracts the relevant data from the Inventory Management System and structures it in a way that facilitates optimal access. In this scenario the Population application pattern primarily deals with structured data.
As another example, consider a financial services portal that aggregates securities analysis from multiple sources and categorizes such information into different folders. In this scenario, the population step involves crawling selected Web sites for specified information, creating an index of selected articles and categorizing them. This example identifies the need for a population step in aggregating and distilling meaningful information from unstructured data.
The patterns for population can be found under Application Integration::Data Integration on this web site.
User Information Access
User Information Access involves designing and creating the user interface and processes for unraveling relevant information from raw data to meet the business needs of the user. User Information Access applications cover a wide range of functions, from simple queries to complex data mining.
The specific business functionality supported by applications that automate the Information Aggregation business pattern vary from one industry to the other. Yet a closer survey of such applications in multiple industries reveals certain common approaches that have been successful. The following Application patterns document these repeatable, successful solutions.
Information Aggregation User identification
Anyone involved in decision-making processes can use applications built according to the Information Aggregation pattern. Often users access data in a read-only format to inform themselves for decision-making tasks. Sometimes they create new data to explore alternative scenarios.
Users of data might be within an organization or external to it. Internal users include executives, managers, and business analysts. These users access data to analyze the long-term performance of a business. Managers of operational departments, marketing and sales personnel, call-center employees, and others all use informational systems to make judgments on short to medium-term business actions. Increasingly, internal data users include employees who work off site and connect to company data using the Internet, an extranet, or through a dial-in mechanism.
External users include customers, partners, agents, and others who are given access to portions of company data to help them in their interactions with the company. Personal customers connect to this data using the Internet. Partners and agents might use an extranet connection to this data for added facilities and security.
The Information Aggregation Usage page contains more information on the business uses of information as facilitated through the Information Aggregation Pattern.
Identification and storage of structured data
While in theory the data needed by users of an Information Aggregation solution includes all a company's data, best practice dictates that the data required for informational purposes be copied into a separate environment from the enterprise's operational data and structured according to the needs of the information access environment. The entire set of such data is usually distributed widely within the information technology systems of the company and on workstations connected to the company's IT systems directly or through intranets, extranets, or the Internet.
The contents, structure, placement, and relationships of data stores (often called the data architecture of the environment) are the key design points for Information Aggregation applications. Best practice data architecture identifies the following key types of data stores and relates them as shown.
In the data architecture depicted above, business data, sourced from operational systems and external data sources, is stored in three distinct types of data store:
- Business data warehouse: This is a data store containing detailed, reconciled, and historical data, structured according to an enterprise data model and designed to be the single, consistent source for all management information. This data source is seldom accessed by end users, and then only in a read-only format.
-
Data mart:
This is a data store defined and designed to meet the information needs of a department or group of users, containing the needed data, either detailed or summarized, and structured according to the query or reporting needs of the user. The data mart is the primary source of information for end users. The data in the data mart is accessed most often in read-only formats, but occassionally in a read-write format.
Data marts come in a variety of forms, including relational databases, dimensional databases, spreadsheets, and others. An increasingly important form of data mart is the Web mart, where data is stored in a Web page or similar format directly usable by Web browsers. - Operational data store: This is a data store containing detailed, partially reconciled, and nearly current data used for immediate reporting needs. Users can often write additional data to this form of data store.
Additionally, metadata (descriptive data about the business data and applications) is stored in the business warehouse catalog.
Functionality and Application Needs
In addition to defining the data architecture, we need to define the functionality required both to create and manage these data stores and to allow the users to access and use the data contained in them.
The specific operations users perform within a business determine the functionality required to access and use data. The function a user requires is often called an application, and will be termed the Business Intelligence (BI) application hereafter. The black arrows in the figure above represent the BI application. BI application functionality covers a wide range of functions, from simple queries to complex and comprehensive applications.
The Information Aggregation Application Classes page contains additional information on the types of applications that can be built using the Information Aggregation pattern.
Recommended reading
Data Warehouse - from Architecture to Implementation, (1997) Addison Wesley, by IBM Distinguished Engineer Dr. Barry Devlin, provides a detailed and comprehensive description of a data warehouse architecture and recommendations for implementing it.
What's Next
If you've determined that the Information Aggregation business pattern can provide an appropriate solution design for your business need, the next step is to select an Application pattern. The Information Aggregation business pattern can be implemented using the base Application pattern or its two variations, providing solution flexibility so that the determined pattern can address the specific needs of the business process being automated. The next step is to select a User Information Access application pattern. If your choice indicates the need for a derived data store (e.g. data warehouse, data mart etc) or index you will later probably want to review the Application Integration::Data Integration application patterns.