Configuring a Kafka handler

How to configure a Kafka handler.

Before you begin

Configure a Kafka cluster in the DataPower® domain. For details, see Configuring a Kafka cluster.

About this task

A Kafka handler monitors the request topic for incoming client requests. To configure a Kafka handler, define the following properties.
  • Which Kafka cluster to associate.
  • The request topic to monitor for incoming client requests. On receipt, the handler forwards the extracted message to the DataPower service for processing. You can validate messages on the request topic against an Avro schema in the local: directory.
  • Because the Kafka server might not expect responses, the response topic to send responses after DataPower processing. When not configured, processed messages are dropped. You can validate messages on the response topic against an Avro schema in the local: directory.
  • The consumer group for retrieving messages.
  • The number of messages that the handler processes as a batch. The handler attempts to retrieve the number of specified messages from the consumer and processes these messages as a batch. This setting is meaningful only when the Kafka cluster is configured to commit offsets to the offset store at the defined interval.

Procedure

  1. In the search field, enter kafka.
  2. From the search results, click Kafka handler.
  3. Click Add.
  4. Define the basic properties - Name, administrative state, and comments.
  5. From the Kafka cluster list, select a Kafka cluster.
  6. Define the request topic that can be schema-validated. By default, request topics are not schema-validated.
    1. In the Request topic field, enter the name of the request topic to monitor for incoming client requests. On receipt, the handler forwards the extracted message to the DataPower service for processing.
    2. To validate the message against a schema, select Avro from the Validate request list.
    3. In the Request schema file field, specify the schema file in the local: directory.
  7. Optional: Define the response topic that can be schema-validated. By default, request topics are not schema-validated.
    1. In the Response topic field, enter the name of the response topic to send responses after processing. Provide a value only when the Kafka server expects responses. When blank, responses are dropped.
    2. To validate the message against a schema, select Avro from the Validate response list.
    3. In the Response schema file field, specify the schema file in the local: directory.
  8. In the Consumer group field, enter the name of the consumer group for retrieving messages from the Kafka server.
  9. In the Batch size field, enter the number of messages that the Kafka handler processes as a batch. The handler attempts to retrieve the number of specified messages from the consumer and processes these messages as a batch.
  10. Click Apply to save changes to the running configuration.
  11. Click Save to save changes to the persisted configuration.

What to do next

Associate the handler with a DataPower service.