Configuring a potential overlay workflow

Configure IBM Master Data Management to recognize and quarantine any potential overlay issues in your master data to prevent potentially breaking data changes before they impact your entities.

A potential overlay issue occurs when a record update results in changes that appear to alter the identity of the record, such as a substantially different name, date of birth, or identification number. This type of change can potentially indicate that the record was erroneously overwritten with data from another record, and could improperly alter entity information.

When a record gets updated, the potential overlay workflow compares the updated demographic data with its current value. If key data elements such as name, phone number, gender, or personal identification numbers are very different, the workflow flags that record as having a potential overlay.

When configuring your workflow, you can define the relevant key data elements and the degree of change that results in a potential overlay issue. When a record update occurs that goes beyond the allowed change level, IBM Master Data Management locks the record to prevent it from being updated further and creates a potential overlay task.

When a record is locked, there are several restrictions that remain in place until the potential overlay task is completed:

  • The record cannot be deleted.
  • Relationships involving the record cannot be created, deleted, or edited.
  • The record cannot be unlinked from an entity or moved to another entity.
  • Entities containing a locked record cannot be joined to other entities.
  • The record cannot be updated by using the IBM Master Data Management workspace interface.
  • Any updates to the record coming from source systems will be acknowledged, but not applied. Instead, these updates will be queued and must be reviewed as part of the remediation task.

From the task inbox, a data steward can remediate each potential overlay issue by reviewing the incoming record updates. During remediation, data stewards must either apply or discard each update that occurred during the locked period, including the initial change that triggered the workflow. By completing the task, the data steward can unlock the record. For more information about working with potential overlay tasks, see Remediating potential overlay issues.

Set up potential overlay workflow task type configurations from the Task types task types icon page. To configure a potential overlay workflow type:

  1. Define the scenarios and conditions that will generate a potential overlay task, either by using the IBM Master Data Management service interface or the IBM Master Data Management API.
  2. Assign data steward users and activate the workflow.
Required permissions
To configure master data workflows, you must be a member of the DataEngineer user group for the IBM Master Data Management service.

Defining a new potential overlay workflow task type

Before your IBM Master Data Management service can start recognizing potential overlay issues, you must set the conditions that trigger the workflow by defining a new potential overlay workflow task type.

To define a new potential overlay workflow task type from within the IBM Master Data Management service interface:

  1. From the Master data navigation menu, click Task types task types icon to open the task types page.

  2. Select Potential overlay tasks to show a list of all existing potential overlay configurations.

  3. If you have any existing configurations, review them here. Optionally, you can select a configuration to edit it.

  4. Click New configuration to start a new potential overlay configuration.

  5. Select the record type that this configuration applies to. You can only create one potential overlay configuration for each record type, but you can define different scenarios within that configuration.

  6. Provide a meaningful description that explains the reason for this configuration.

  7. Define whether this configuration applies to all data sources or only specific data sources. You might only want to create potential overly tasks if the records come from specific data sources.

    If you select Specific sources, then you must choose the sources from the list of available data sources in your IBM Master Data Management instance. If you cannot find a particular source in the list, you can create a new one by clicking New source. The new source won't have any assets connected to it until an asset from that source gets added to IBM Master Data Management.

  8. Define one or more scenarios that will cause this type of potential overlay task to be created.

    1. Provide a name for the scenario.
    2. Define the conditions that will trigger a potential overlay task. Select a specific attribute from your data model, and then define the comparison distance that will trigger the task creation.

    Comparison distance is a measure of how different a value is compared to another value, in terms of the number of different characters. For example, a distance of 2 indicates that two characters in the values are different.

    It is important to carefully consider the condition settings. The names Bobby and Robby have a comparison distance of 1, which is a minor difference. However, the ID numbers 123 432 812 and 123 433 812 also have a comparison distance of 1, but this can indicate a major problem with a record.

    If a condition's chosen attribute is not also selected as a matching attribute, then IBM Master Data Management cannot calculate the comparison distance and the condition will be ignored. For information about selecting matching attributes, see Matching your data to create master data entities.

    1. Add as many conditions as necessary to define this scenario.
    2. Define whether this scenario applies to all data sources or only specific data sources. You might want to handle potential overlay tasks differently if they come from trusted (or non-trusted) sources.

    The list of available sources depends on the data sources that you selected when initially creating the parent configuration of this scenario.

    1. Click Create to save your scenario.

    Define as many scenarios as required to configure potential overlay task creation for this record type across all of the necessary data sources. When you are done defining sources, click Next.

  9. Choose a platform-level workflow configuration to associate with this potential overlay configuration, or create a new workflow configuration. You must associate a platform workflow configuration to enable potential overlay tasks to be created. Click Next.

  10. Add one or more rules to automatically apply searchable tags to potential overlay issues that meet your conditions.

    1. Click Add rule.
    2. Provide a name for the autotagging rule.
    3. Define the conditions that will trigger this autotagging rule. Click Add condition to define as many conditions as necessary.
    4. Select one or more tags that will be applied to issues that meet the conditions. To create a new tag, click Create new tag, then define the tag content and appearance and click Save when you are done. For more information about creating tags, see Managing master data tags.
    5. Click Create.

    To edit an autotagging rule, click the Edit icon edit icon. To delete an autotagging rule, click the Delete icon delete icon.

    Important: When you edit or delete a rule, your previously applied tags are not affected. Changes to rules take effect immediately, but the new rules only affect future tagging. Deleted rules do not result in deletion of the corresponding tags.

    When you are done defining autotagging rules, click Next.

  11. Review your configuration details, then click Create to complete this potential overlaw configuration.

When you are ready to activate this workflow configuration, continue to the next task, Assigning users and activating your workflow configuration.

Assigning users and activating your workflow configuration

After you have defined the data sources, scenarios, and conditions that make up your potential overlay workflow configuration, you can assign data steward users and activate the configuration by using the platform's workflow configuration tools.

To configure how users get assigned potential overlay tasks and activate the workflow:

  1. From the Master data navigation menu, click Task types task types icon to open the task types page.

  2. Select Potential overlay tasks to show a list of all existing potential overlay workflow task configurations.

  3. Select the existing task configuration that you want to activate, then choose Edit. Click Next to advance to the workflow configuration step.

  4. Click New workflow configuration to define what users should be assigned tasks.

  5. Provide a name for the workflow configuration, then click Create.

    When the configuration is created, it is inactive by default. Inactive workflow configurations capture potential overlay issues, but do not assign tasks to users.

    It is a good idea to monitor how the configuration performs before activating it.

  6. Expand the workflow configuration, then open the workflow configuration settings. The workflow configuration page opens in the platform interface.

    1. Select the data steward users who should be assigned the tasks created by this workflow.
    2. Define how users should be notified about each type of event.
    3. Click Save to save your changes without activating the configuration or, if you are ready to activate this workflow, click Activate to save and activate the configuration.

    For more information about configuring workflows, see Managing workflows

  7. Close the platform browser tab and return to the IBM Master Data Management workflow configuration page, then click Refresh beside the configuration that you have activated. If you activated the workflow, the row should now show as Active. Click Next.

  8. Review your configuration details, then click Create to complete this potential overlaw workflow configuration.

Defining a new potential overlay workflow configuration by using the API

Before your IBM Master Data Management service can start recognizing potential overlay issues, you must set the conditions that trigger the workflow by defining a new potential overlay workflow task type. As an alternative to using the service interface, you can use API commands to define the workflow.

To configure a new potential overlay workflow task type by using the API, use the PUT workflows_configuration command from the model microservice API:

PUT /mdm/v1/workflows_configuration/potential_overlay

In the payload of this API, you must define the following:

  • What record types are applicable for the potential overlay workflow
  • What data sources to consider
  • What attributes to consider
  • What conditions qualify a given change to be considered a potential overlay

Here is a sample payload:

{
  "description": "Potential overlays configuration",
  "auto_create_tasks": true,
  "applicable_data_types": {
    "record_types": ["person"]
  },
  "trigger_operator": {
    "record_types": {
      "person": {
        "name": "Configuration for person record type",
        "entity_types": {
          "person_entity": {
            "triggers": {
              "default": [
                {
                  "recipe": {
                    "date_compare": 2,
                    "pername_compare": 2
                  },
                  "label": "Name and DOB",
                  "status": "active"
                }
              ],
              "source_based": {
                "sourceB": [
                  {
                    "recipe": {
                      "identifier_compare": 2
                    },
                    "label": "Identification Number",
                    "status": "active"
                  }
                ],
                "sourceC": [
                  {
                    "recipe": {
                      "gender_compare": 3,
                      "pername_compare": 3
                    },
                    "label": "Name and Gender",
                    "status": "inactive"
                  }
                ]
              }
            }
          }
        },
        "excluded_sources": ["sourceD"],
        "included_sources": [],
        "workflow_configurations": {
          "default": {
            "name": "default platform workflow configuration",
            "id": "4c611af9-0bf3-4658-ae7f-ade12446685d"
          },
          "source_based": {
            "sourceA": {
              "name": "platform workflow configuration for sourceA",
              "id": "fd91a479-2559-45f3-a21f-0abec1fdf723"
            }
          }
        },
        "status": "active"
      }
    }
  }
}

For more information about using the API, see the IBM Master Data Management API reference documentation.

Learn more