Using Kafka with IBM App Connect Enterprise

You can use the IBM® App Connect Enterprise Kafka nodes to produce and consume messages on Kafka topics.

Before you begin

About this task

You can use the KafkaProducer node to publish messages that are generated from within your message flow to a topic that is hosted on a Kafka server. The published messages are then delivered by the Kafka server to all topic consumers (subscribers). You can use a KafkaConsumer node in a message flow to subscribe to a specified topic on a Kafka server. The KafkaConsumer node then receives messages that are published on the Kafka topic, as input to the message flow. You can also read an individual message that is published on a Kafka topic, by using a KafkaRead node in a message flow to specify the offset position of the message in the topic partition.

To see the version of Kafka that is supported by your installation of IBM App Connect Enterprise 12, complete the following steps:
  1. Go to the IBM support page for the version of IBM App Connect Enterprise 12 that you installed. For example:
  2. Expand the section Embedded components and versions included in this release to display a table of values.
  3. In the column Embedded component, search for the entry for Kafka.
  4. In the column Versions included in this release, read the corresponding value for Kafka. For example, in IBM App Connect Enterprise 12.0.1, the supported version of Kafka is 2.8.0.

For more information about Kafka version compatibility, see the Apache Kafka documentation.

Procedure

Use the KafkaProducer, KafkaConsumer, and KafkaRead nodes to complete the following tasks:

For more information about configuring and using Kafka nodes, see the following topics: