Designing jobs that use the Kafka connector

You can use the Kafka Connector stage in your jobs to read messages from the topic(s) or write messages to the topic(s) in Kafka.

Before you begin

Procedure

  1. Design a job that contains the Kafka Connector stage.
  2. Define a connection to Kafka data source.
  3. To set up the Kafka Connector stage as a source stage to read messages from the topics residing in Kafka, complete the following steps
    1. Configure the Kafka Connector stage as a source.
    2. Set up column definitions on the output link.
    Note: The Kafka Connector stage does not support reject link.
  4. To set up the Kafka Connector stage as a target stage to write data from the topics in Kafka, complete the following steps
    1. Configure the Kafka connector stage as a target.
    2. Set up column definitions.
  5. Compile and run the job.