Advanced master data export using the IBM Match 360 REST API

As an alternative method of exporting master data to a file, you can use the IBM Match 360 REST API. By using the export API, you can apply advanced options beyond the capabilities of the master data workspace's Search tab.

Use the export API to:

  • Export entity data from a specified point in time instead of exporting the full data set.
  • Include information about deleted entities since a specified point in time.

In this topic:

Exporting master data from a specific point in time

The ability to refine exported data based on the last updated timestamps (entity_last_updated or record_last_updated) provides your organization with a way to more efficiently synchronize entity information with downstream systems. Instead of having to resynchronize the full downstream data set, you can use a smaller exported data set that captures the changes since the last synchronization.

This capability also complements configurations that use ongoing data streaming to downstream systems. In case of streaming failures, you can now "catch up" your downstream entity data in a bulk manner so that changes made during a streaming downtime aren't lost.

Exporting information about deleted records or entities

When you export record or entity data from a specific point in time, you can also choose to include information about deleted records or entities since that time.

If you include information about deleted records or entities in your data export, the generated data file includes an additional column: isActive. For deleted records or entities, the isActive column has a value of false.

Deleted records only have values for the recordId, recordType, and recordLastUpdated columns in the exported data. All other columns are blank.

Deleted entities only have values for the entityId, entityType, and entityLastUpdated columns in the exported data. All other columns are blank.

Exporting master data by using the export API

To initiate a master data export job by using the REST API, use the data_exports endpoint in the data-ms microservice of IBM Match 360.

POST /mdm/v1/data_exports

The data_export payload accepts the following parameters:

Parameters of the data_export API
Parameter Description
export_type The type of data being exported in this job: entity or record. If the include_deletes flag is true, then only entity is supported.
file_name The name of the export file to be created.
format The format of the exported data: CSV, TSV, or JSON.
search_criterai.include_deletes Defines whether the exported data includes deleted entities: true or false. If this flag is true, then you must also include a query expression based on the enitity_last_updated field.
search_criteria.joins Defines whether the exported data includes information about join actions: true or false.
search_criteria.query Defines the list of expressions and the connection operator for this data export query. If the includes_deletes flag is true, then you must include a query expression based on the enitity_last_updated field, and cannot provide any other fields in the query.
search_criteria.search_type The type of data to export: entity or record. If the entity type being exported is configured to persist in the database, or if the include_deletes flag is true, then only entity is supported.
search_criteria.filters Define a filter for the export job. This limits the valid entity types in the exported file. For example, a value of person_entity will only export Person entities from the system.

For example data_export API payloads, see the Entity payload example and the Record payload example.

After running the data_export API endpoint, you can monitor the progress of your export job by clicking the Jobs in progress icon in the master data configuration interface.

Entity data_export payload example

Here is an entity type example payload for the data_export API with the include_deletes flag set to true:

{
  "export_type": "ENTITY",
  "format": "CSV",
  "search_criteria": {
    "query": {
      "expressions": [
        {
          "property": "entity_last_updated",
          "condition": "GREATER_THAN_EQUAL",
          "value": "<timeForExport>",
          "expressions": [
            
          ]
        }
      ]
    },
    "filters": [
      {
        "type": "ENTITY",
        "values": [
          "person_entity"
        ],
        "data_quality_issues": [
          
        ]
      }
    ],
    "joins": [
      
    ]
  },
  "file_name": "<filename>",
  "include_deletes": true
}

Record data_export payload example

Here is a record type example payload for the data_export API with the include_deletes flag set to true:

{
  "export_type": "RECORD",
  "format": "CSV",
  "search_criteria": {
    "query": {
      "expressions": [
        {
          "property": "record_last_updated",
          "condition": "GREATER_THAN_EQUAL",
          "value": "<timeForExport>",
          "expressions": []
        },{
          "property": "record_source",
          "condition": "equal",
          "value": "<recordSource>"
        }
      ],
      "operation": "and"
    },
    "filters": [
      {
        "type": "RECORD",
        "values": [
          "person"
        ],
        "data_quality_issues": []
      }
    ],
    "joins": []
  },
  "file_name": "<filename>",
  "include_deletes": true
}

Persisted entity type payload example

Here is an example payload for when the entity type is configured to persist in the database:

{
    "file_name": "PersonEntity",
    "format": "csv",
    "search_criteria": {
        "search_type": "entity",
        "query": {
            "operation": "and",
            "expressions": [{
                "value": "*"
            }]
        },
        "filters": [{
            "type": "entity",
            "values": ["person_entity"]
        }]
    },
    "export_type": "entity"
}

Learn more

Parent topic: Exporting master data from IBM Match 360