Scenario: Tracking and managing exception sets

This scenario describes how the fictional Sample Outdoor Company reviews, tracks, manages, and resolves exceptions that are generated by data rules.

The fictional Sample Outdoor Company sells and distributes products to third-party retailer stores and consumers. The company has consolidated data from several acquisitions into their primary database. To understand, cleanse, and transform the data, the company sets up projects in IBM® InfoSphere® DataStage®, IBM InfoSphere QualityStage®, and IBM InfoSphere Information Analyzer.

In InfoSphere Information Analyzer, the company created the InvalidTaxID data rule. The InvalidTaxID rule evaluates data records to determine whether the data in the TAXID column is a valid nine-digit tax identification number.

The company now wants to monitor the exceptions that do not meet the conditions of the rule. To monitor these exception sets, the company sets up and uses the event and exception management components:

Exception management reports in IBM InfoSphere Information Governance Dashboard
Data Quality Exception Console
IBM Stewardship Center

Reviewing exceptions and assessing the quality of your data with the exception management reports of InfoSphere Information Governance Dashboard

To review exception sets that are generated when the rule is run, employees of the fictional Sample Outdoor Company complete the following steps:

The data steward manager begins by opening an exception management workspace in IBM Cognos Connection, selects a scope of interest that is based on the project that contains the InvalidTaxID data rule. The data steward manager then selects Data Steward Manager Summary to see charts with information about those exception sets.
The data steward manager selects a bar in one of the charts to drill down to the corresponding exception set report. To review details of the exception set, the data steward manager clicks the name of the exception set, which opens the Exception Set Details report. On the Exception Sets report page, the data steward manager clicks the link to the Data Quality Exception Console to further assess the exception set.

Monitoring exception sets and viewing exception records in the Data Quality Exception Console

To monitor exceptions that are generated when the rule is run, employees of the fictional Sample Outdoor Company complete the following steps:

In the Data Quality Exception Console, the data steward manager filters to find the InvalidTaxID exception set in the list of new exception sets and assigns it high priority.
The data steward manager wants to send that exception set along with all subsequent exception sets produced by that data rule to be managed in the IBM Stewardship Center. The data steward manager selects the set and clicks Manage in the Data Quality Exception Console to send the exception sets to the Data Quality Exception sample process application in IBM Stewardship Center.

Managing exception sets with the IBM Stewardship Center

To manage exceptions sets that are sent to the Data Quality Exception sample process application in IBM Stewardship Center, employees of the fictional Sample Outdoor Company complete the following tasks:

In the Work dashboard of IBM Stewardship Center, the InvalidTaxID exception set displays as a task in the data steward team task list.
A data steward from the data steward team logs in to IBM Stewardship Center and clicks the task to claim it. The data steward is able to view details about the exception set that can help to determine how to resolve the data quality issue. When the task is claimed in IBM Stewardship Center, the owner and status of the exception set are updated and displayed in the Data Quality Exception Console.
The data steward collaborates with the stewardship team by using the collaboration tools in IBM Stewardship Center and exports the exception records to a .csv to make a batch of edits to the data. The data steward imports the fixes back into the system, and clicks Submit Fixes to send the fixes to the data steward manager for review. The owner and status of the exception set changes again and is displayed in the Data Quality Exception Console.
The data steward manager logs in to IBM Stewardship Center, sees the fixed records as a task in the task list, and clicks the task to claim it. IBM Stewardship Center displays the fixes, attachments, and notes that the data steward added to the exception set. After reviewing the fixes, the data steward manager clicks Approve Fixes to accept the them and send the approved fixes to the data source owner team to be applied to the original data source. The owner and status of the exception set changes again and is displayed in the Data Quality Exception Console
The data source owner logs in to IBM Stewardship Center, sees the approved fix records as a task in the task list, and clicks the task to claim it. IBM Stewardship Center displays the fixes, attachments, and notes that the data steward and data steward manager added to the exception set. After reviewing the fixes, the data source owner makes the appropriate changes to the original data source and then clicks Apply Fixes. The completed task is removed from the task list. The owner and status of the exception set changes again and is displayed in the Data Quality Exception Console
An InfoSphere Information Analyzer user runs the same rule with the new input records in place to confirm the fixes.