Resolving The Problem

Review the health of the Kafka topic:

• Review the topic has all replication factors and leaders to all partitions.

  • bin/kafka-topics.sh --describe --zookeeper localhost:2181 --topic <topic_name>

• Try to write few messages to the same topic from outside of SDC with kafka-console-producer:

  • bin/kafka-console-producer.sh --broker-list localhost:9092 --topic <topic_name>

If network side of things looks good then another possibility is that records are coming in at a faster rate into the queue before the client can send those to Kafka. There are few knobs you can turn to address this on Kafka side. You can increase the batch.size, and if you do that you'll need to increase the buffer.memory as well. We would suggest trying increasing request.timeout.ms first and before making any changes, please check with your Kafka vendor to fully understand the impact of these changes. 

• Also, you could try setting up the retries and retry.backoff.ms in producer configurations. Both of these properties, you can configure in the Kafka Producer destination --> Kafka tab --> Kafka Configuration.

Kafka documentation