Adding data and mapping it to your data types

Adding record data from a flat data file

To add record data into IBM Master Data Management from a CSV or TSV data file:

1. From the Master data navigation menu, click the Assets icon .

2. In the action bar of the Assets page, click Add asset.

3. From the Data assets panel, choose whether to add a data asset by upload, from the project, or from the catalog. To upload a data file, choose Load.

   For information about adding an asset from a project, see Adding data or sources through your project.

4. On your local computer, select a flat data file in CSV or TSV format and drag it into the Data assets panel.

   When the file finishes uploading, the asset automatically opens so that you can begin mapping its data columns to the data types defined in your data model.

5. After you first load an asset, you must tell the system what type of content it contains, such as a specific record type. It is important to assign the correct content type to each asset so that IBM Master Data Management can find the data type that best fits the data. Assets in your asset list that do not have their asset content type defined have a warning status that says Set asset content. You cannot map an asset without first defining its content type.

   To define an asset's content type:

   1. Select the asset in the Asset list panel and then click Set asset content.
   2. In the dialog that opens, select the correct data type for this asset. If you cannot find an appropriate data type in the list, then you might have to customize your data type definitions. For more information, see Customizing your data types.
   3. Optionally, if token-based access control is enabled for this IBM Master Data Management deployment, select a token to assign to the data contained in this asset. If you cannot find an appropriate token in the list, then you might have to create a new access token. For more information, see Configuring token-based access control.
   4. Click Save.

Next step: Map data columns to your data types

Adding data or sources through your project

You can add data assets, sources, or connections to IBM Master Data Management through your project.

You can use IBM Master Data Management as a connected data source or target. For information about setting up and using the IBM Master Data Management connection, see IBM Master Data Management connection.

Any data asset files that you want to load into IBM Master Data Management must include a file extension of a supported type, such as .csv or .tsv. This requirement includes assets that are already in your project. If an asset name in your project does not include a supported file extension, it will appear greyed-out when you try to add it. To edit the name of an asset in your project, go to the Assets page, select your asset, then edit the name in the About this asset panel.

Watch this video to see how to create a connection and add connected data to a project.

This video provides a visual method to learn the concepts and tasks in this documentation.

• Transcript
  Synchronize transcript with video

  Video transcript

  Time	Transcript
  00:00	This video shows you how to set up a connection to a data source and add connected data to a project.
  00:08	If you have data stored in a data source, you can set up a connection to that data source from any project.
  00:16	From here, you can add different elements to the project.
  00:20	In this case, you want to add a connection.
  00:24	You can create a new connection to an IBM service, such as IBM Db2 and Cloud Object Storage, or to a service from third parties, such as Amazon, Microsoft or Apache.
  00:39	And you can filter the list based on compatible services.
  00:45	You can also add a connection that was created at the platform level, which can be used across projects and catalogs.
  00:54	Or you can create a connection to one of your provisioned IBM Cloud services.
  00:59	In this case, select the provisioned IBM Cloud service for Db2 Warehouse on Cloud.
  01:08	If the credentials are not prepopulated, you can get the credentials for the instance from the IBM Cloud service launch page.
  01:17	First, test the connection and then create the connection.
  01:25	The new connection now displays in the list of data assets.
  01:30	Next, add connected data assets to this project.
  01:37	Select the source - in this case, it's the Db2 Warehouse on Cloud connection just created.
  01:43	Then select the schema and table.
  01:50	You can see that this will add a reference to the data within this connection and include it in the target project.
  01:58	Provide a name and a description and click "Create".
  02:06	The data now displays in the list of data assets.
  02:09	Open the data set to get a preview; and from here you can move directly into refining the data.
  02:17	Find more videos in the Cloud Pak for Data as a Service documentation.

For more information about adding data directly to your project, see Adding data to a project.

After adding data, you must set your asset content type:

1. Select the asset in the Asset list panel and then click Set asset content.
2. In the dialog that opens, select the correct data type for this asset, then click Save. If you cannot find an appropriate data type in the list, then you might have to customize your data type definitions. For more information, see Customizing your data types.

Next step: Map data columns to your data types

Mapping data columns to your data types

Before you can publish a data asset to be used in IBM Master Data Management, you must map it. Each of an asset's columns must either (a) be mapped to a corresponding attribute or field in your data types, or (b) be excluded from mapping.

To map a data asset, you have several options:

• You can manually map each column.
• You can map each column with the assistance of profiling and automapping.
• You can apply a mapping pattern.

Remember: To use the profiling and automapping features of IBM Master Data Management, your IBM Cloud Pak for Data deployment must include IBM watsonx.data intelligence.

For details about manually mapping or using automapping, read the following procedure. For details about applying a mapping pattern, see Applying a mapping pattern to a data asset.

To map a data asset to your data types:

1. From the Master data navigation menu, click the Assets icon .

2. From the Asset list panel, select the data asset that you want to map into the system. The asset's data columns will load into the mapping panel.

   To help you to find the asset you are looking for, you can search by asset name or filter based on the number of columns, record type, mapping status, publishing status, or available mapping patterns. Click the Filter icon to apply a filter.

   The data from the asset you select displays in tabular format with a number of rows and columns. Each column represents an attribute that must be mapped to a corresponding attribute type in the data type definitions. When you first open a data source or asset, each column is marked with a Not Mapped tag.

   Tip: You can manually map each column if you choose, but you can greatly speed up the mapping process by taking advantage of the automapping feature.

3. Click the info icon to open the About this asset panel. In the panel, review the mapping details and statistics for this asset. At a glance, you can see how many data columns from this asset have been mapped, if any.

4. To manually map a column, select it, then use the Mapping targets panel that opens to search for and select the appropriate attribute or field from your data type definitions. Click Map and save to data model.

   If an appropriate attribute or field does not exist in the data type definitions, you can create one from the Mapping targets panel. Click either New > Create attribute or New > Create field to define and provide the details of a new attribute or field.

   If you choose to create a Simple attribute instead of assigning an existing attribute type, then the new simple attribute is added directly to that record type's data model. It is not categorized under Attribute types.

5. To exclude a column from mapping, click on the column to open the Mapping targets panel, then click Exclude column.

   If you need to exclude more than one column, click the Exclude columns icon beside the search bar or on the Mapping targets panel. Select the columns you want to exclude, then click Save.

6. To enable automapping for this source or asset, you must first profile the data. Click Profile data.

   Profiling analyzes and classifies your data to enable the automapping process to take place. Profiling can take some time to complete, so it runs in the background so that you can continue working. You might want to start reviewing and manually mapping some columns.

   Automapping will never overwrite any manual mapping that you have done.

7. When profiling completes, click Automap asset. IBM Master Data Management analyzes your data and automatically maps as many columns as possible to the data type definitions. Even if it cannot map a specific column, the automap function can suggest some of the most likely mapping selections.

8. Review the automapping results. If any of the mappings are incorrect, or if a column remains unmapped, then manually map it correctly. Alternately, if a specific column is not required, you can exclude it from being loaded into IBM Master Data Management by selecting Exclude column.

9. Scroll horizontally through the columns to ensure that every column in your data source or asset is mapped. If any columns are not mapped, automapped, or explicitly excluded from mapping, then the data asset remains in a Partially mapped state.

   Important: If the record_source attribute is mapped to any field in the asset, then that field must be populated in all of the asset's records. If any record is missing a value for the record_source field, then the asset cannot successfully load. If you do not map the record_source attribute to an existing field, then a default record source name is derived using the asset name.

10. When you finish mapping the data asset, you're ready to publish the data into the system.

    • If your data types are new or have changed, publish the data types by clicking the publish data types icon in the action bar. Wait for the publish job to complete.
    • To publish your data after publishing your data types, click the publish data icon in the action bar. Wait for the publish job to complete.

Applying a mapping pattern to a data asset

Mapping patterns help you to maintain consistency across similar data assets by making it easy to repeat your data mapping selections for compatible assets.

A mapping pattern is automatically created when you manually map a data asset. The pattern saves your column mapping selections so that they can be reused by other data assets that share the same column format and record type. By applying a mapping pattern, you can avoid manually mapping data assets that are similar to existing assets that you have already mapped.

IBM Master Data Management identifies when a new asset is compatible with an existing mapping pattern in the system and then notifies you that you can use a pattern to avoid manual mapping work.

Mapping patterns can be created by manually mapping data assets or they can be imported by using configuration snapshots.

For information about managing and applying mapping patterns by using configuration snapshots, see Saving and loading configuration snapshots.

To apply a mapping pattern to a mapped or unmapped data asset:

1. From the Master data navigation menu, click the Assets icon .

2. From the Asset list panel, locate one or more data assets that you want to apply a mapping pattern to.

   To help you find the assets you are looking for, you can search by name or filter based on the number of columns, record type, mapping status, publishing status, or available mapping patterns. Click the Filter icon to apply a filter.

3. Select the data assets that you want to apply a mapping pattern to.

   • To apply a mapping pattern to a single data asset, click the Apply mapping pattern icon next to the asset name.
   • To apply a mapping pattern to one or more data assets, select the checkbox next to the asset name, then click Apply mapping in the Assets list. The selected data assets must share the same structure and column format to be able to share a mapping pattern.

   The Apply mapping patterns page shows a list of available mapping patterns. If there is a recommended matching pattern for this asset, it has a badge icon next to it.

4. Review the mapping patterns. You can see what other assets the mapping pattern currently applies to, along with the applicable record type, last updated date, and original source.

   For more details and to compare mapping patterns to each other, select a primary mapping pattern, then click Compare mapping patterns. Horizontally scroll through the patterns to compare them to the one that you selected. Vertically scroll on the page to view more details such as snapshot details and column mappings.

5. Select the mapping pattern that you want to apply to the selected data assets, then click Next.

6. Review the mapping changes that you have selected. Confirm that you have chosen the correct assets and mapping pattern.

   Applying a mapping pattern can change an asset's record type. Be careful not to erroneously change the record type. Mapping changes done by applying a mapping pattern cannot be undone without manually remapping the asset.

7. Click Finish to apply the mapping pattern.

8. After applying the mapping pattern, you're ready to publish the data into the system.

   • If your data types are new or have changed, publish them first by clicking the publish data types icon in the action bar. Wait for the publish job to complete.
   • To publish your data after publishing your data types, click the publish data icon in the action bar. Wait for the publish job to complete.

Adding relationship data from a flat data file

Before you can load a relationship data asset into IBM Master Data Management, you must first define the corresponding relationship type in the data type definitions. For details, see Customizing your data types.

Relationship data assets are formatted into delimited rows (CSV or TSV). There are several required data columns:

• Record IDs for both parties in each relationship
• Record types for both parties in each relationship
• Record sources for both parties in each relationship

You can create the relationship data asset manually, through an ETL process, or by using the application where your relationships are stored.

To add relationship data into IBM Master Data Management from a CSV or TSV data file:

1. From the Master data navigation menu, click the Assets icon .

2. Load, map, and publish the record data assets into IBM Master Data Management. These data assets should contain the record data that you want to associate using relationships. For details, see Adding record data from a flat data file and Mapping data to your data types.

3. Load your relationship data asset file:

   a. In the action bar, click Add asset.

   b. From the Data assets panel, choose whether to add data assets by upload, from the project, or from the catalog. To upload a data file, choose Load.

   Note: If your data includes governed catalogs, you might be unable to view or add some catalog assets. Depending on your permissions, you might only be able to view catalog assets that you own or manage.

   c. On your local computer, select a flat data file containing relationship data in CSV or TSV format and drag it into the Data assets panel. When the file finishes uploading, it is added to your assets list.

4. Define the asset's content type:

   1. In the Asset list panel and then click Set asset content.
   2. In the dialog that opens, select Relationships from list of data types.
   3. Click Save.

   If the appropriate relationship type is not in the asset data type list, then you might have to customize your data type definitions to add it. For more information, see Customizing your data types.

5. Map and publish your relationship data asset. For details, see Mapping data to your data types. Be sure to map each of the required data columns: from record ID, to record ID, from record type, to record type, from record source, and to record source.

   Restriction: Relationship data does not support profiling and automapping. Manually map your columns to the data types.

Next steps

• Customizing your data types
• Add master data published from InfoSphere MDM Standard or Advanced Edition
• Set up and run matching

Learn more

• Saving and loading configuration snapshots
• Working with governed data in IBM Master Data Management
• Master Data Management tutorial: Configure a 360-degree view