Kafka Connect scenarios

With IBM® MQ and Apache Kafka specializing in different aspects of the messaging spectrum, one on connectivity and the other on data, solutions often require data to flow between the two. You can achieve this using Kafka Connect.

Kafka Connect provides a framework for moving data from an external system into a Kafka cluster, or from a Kafka cluster into an external system. This is achieved by connectors.

There are many different types of connectors available, and IBM provides connectors for use with IBM MQ. Connectors come in two different types:

Source connectors transfer data from an external system into Kafka.
The IBM MQ source connector consumes messages from an IBM MQ queue and publishes them as events into a Kafka topic.
Sink connectors transfer data into an external system from Kafka.
The IBM MQ sink connector consumes events from a Kafka topic and sends them as messages to an MQ queue.

See Kafka Connect and connectors for more information.

Kafka Connect scenarios might include:

A core banking system with IBM MQ used as the connectivity backbone. You want to take a copy of messages moving through IBM MQ and push them into Kafka for analytics
You want to extend your core banking system to emit data into Kafka, but only want data to enter Kafka when the banking transaction completes successfully, so use IBM MQ as a transactional bridge
You need to get data into z/OS® from Multiplatforms. Multiplatform development team has experience with Kafka, z/OS team want to exploit IBM MQ integration with CICS® / IMS

From IBM MQ 9.4.0, if your enterprise has IBM MQ Advanced entitlement, IBM MQ Advanced for z/OS VUE entitlement, IBM MQ Advanced for Multiplatforms entitlement, or IBM MQ Appliance entitlement, you get access to IBM provided, and supported source and sink connectors.

Notes:

These approaches can be used with any variant of Kafka, for example, Apache Kafka and IBM Event Streams.
Support is provided only for the two IBM connectors, not the Kafka Connect framework itself.