Introduction to WebSphere QualityStage

Scenarios for data cleansing

Organizations need to understand the complex relationships that they have with their customers, suppliers and distribution channels. They need to base decisions on accurate counts of parts and products to compete effectively, provide exceptional service, and meet increasing regulatory requirements. Consider the following scenarios:

Banking: One view of households

To facilitate marketing and mail campaigns, a large retail bank needed a single dynamic view of its customers’ households from 60 million records in 50 source systems.

The bank uses WebSphere QualityStage to automate the process. Consolidated views are matched for all 50 sources, yielding information for all marketing campaigns. The result is reduced costs and improved return on the bank's marketing investments. Householding is now a standard process at the bank, which has a better understanding of its customers and more effective customer relationship management.

Pharmaceutical: Operations information

A large pharmaceutical company needed a data warehouse for marketing and sales information. The company had diverse legacy data with different standards and formats, information that was buried in free-form fields, incorrect data values, discrepancies between field metadata and actual data in the field, and duplicates. It was impossible to get a complete, consolidated view of an entity such as total quarterly sales from the prescriptions of one doctor. Reports were difficult and time-consuming to compile, and their accuracy was suspect.

Most vendor tools lack the flexibility to find all the legacy data variants, different formats for business entities, and other data problems. The company chose WebSphere QualityStage because it goes beyond traditional data-cleansing techniques to investigate fragmented legacy data at the level of each data value. Analysts can now access complete and accurate online views of doctors, the prescriptions that they write, and their managed-care affiliations for better decision support, trend analysis, and targeted marketing.

Insurance: One real-time view of the customer

A leading insurance company lacked a unique ID for each subscriber, many of whom participated in multiple health, dental, or benefit plans. Subscribers who visited customer portals could not get complete information on their account status, eligible services, and other details.

Using WebSphere QualityStage, the company implemented a real-time, in-flight data quality check of all portal inquiries. WebSphere QualityStage and WebSphere MQ transactions were combined to retrieve customer data from multiple sources and return integrated customer views. The new process provides more than 25 million subscribers with a real-time, 360-degree view of their insurance services. A unique customer ID for each subscriber is also helping the insurer move toward a single customer database for improved customer service and marketing.

Where WebSphere QualityStage fits in the overall business context

WebSphere QualityStage performs the preparation stage of enterprise data integration (often referred to as data cleansing), as Figure 1 shows. WebSphere QualityStage leverages the source systems analysis that is performed by WebSphere Information Analyzer and supports the transformation functions of WebSphere DataStage™.

Figure 1. WebSphere QualityStage prepares data for integration

Working together, these products automate what was previously a manual or neglected activity within a data integration effort: data quality assurance. The combined benefits help companies avoid one of the biggest problems with data-centric IT projects: low return on investment (ROI) caused by working with poor-quality data.

Data preparation is critical to the success of an integration project. These common business initiatives are strengthened by improved data quality:

Consolidating enterprise applications

High-quality data and the ability to identify critical role relationships improves the success of consolidation projects.

Marketing campaigns

Strong understanding of customers and customer relationships cuts costs, improves customer satisfaction and attrition, and increases revenues.

Supply chain management

Better data quality allows better integration between an organization and its suppliers by resolving differences in codes and descriptions for parts or products.

Procurement

Identifying multiple purchases from the same supplier and multiple purchases of the same commodity leads to improved terms and reduced cost.

Fraud detection and regulatory compliance

Better reference data reduces fraud loss by quickly identifying fraudulent activity.

Whether an enterprise is migrating its information systems, upgrading its organization and its processes, or integrating and leveraging information, it must determine the requirements and structure of the data that will address the business goals. As Figure 2 shows, you can use WebSphere QualityStage to meet those data quality requirements with classic data re-engineering.

Figure 2. Classic data reengineering with WebSphere QualityStage

A process for reengineering data should accomplish the following goals:

• Resolve conflicting and ambiguous meanings for data values
• Identify new or hidden attributes from free-form and loosely controlled source fields
• Standardize data to make it easier to find
• Identify duplication and relationships among such business entities as customers, prospects, vendors, suppliers, parts, locations, and events
• Create one unique view of the business entity
• Facilitate enrichment of reengineered data, such as adding information from vendor sources or applying standard postal certification routines

You can use a data reengineering process in batch or real time for continuous data quality improvement.