Overview of the Kafka runtime processor
Use the Kafka runtime stream processor for real-time data synchronization of MDM data with Analytical MDM and other systems in Cloud, On-Premise, and Hybrid scenarios.
The runtime stream processor, also known as the ongoing stream processor, works in conjunction with the MDM Kafka Channel for pushing real-time changes to Kafka topics.
The runtime processor uses a three stage extraction solution:
- The processor pushes the current data change to a Kafka topic using an MDM extension and the Kafka channel.
- The processor sends the consolidated entity ID to another Kafka topic for further processing.
Note: Within InfoSphere® MDM at runtime, the same party can be updated multiple times within a small time window. Kafka provides windowing and aggregation functions to consolidate changes and avoid multiple reads from InfoSphere MDM.
- The consolidated entity IDs are collected and processed for further transformation and persistence to one or more downstream systems.
