Known issues and workarounds

There are situations when Kafka Connector can behave unexpectedly and when manual user intervention is required. This topic provides guidance for such cases.

Error: Commit cannot be completed

Problem: When Kafka connector is configured to run in Continuous mode, the job might fail with the message:

Kafka Connector was unable to commit messages due to following error: Commit cannot be completed since the group has already rebalanced and assigned the partitions to another member. This means that the time between subsequent calls to poll() was longer than the configured max.poll.interval.ms, which typically implies that the poll loop is spending too much time message processing. You can address this either by increasing the session timeout or by reducing the maximum size of batches returned in poll() with max.poll.records.

Solution: Change the connector execution mode to Sequential.

Error: Invalid property value/Usage/MaxMessage

Problem: A job fails with error message: Invalid property value /Usage/MaxMessage. This can happen when you import job that has the Max Messages property that is configured by using job parameter. As the property is removed in recent connector code, DataStage is not able to match the job parameter with current connector configuration properties.

Solution:
  1. Open the job and edit Kafka Connector properties.
  2. Modify any of the stage parameters. It is not required to change particular value. Editing any of the configuration properties trigger stage recompilation.
  3. Compile a job.
  4. Run a job. Error must be gone.

Job log with warning: java.io.IOException: DER input, Integer tag error

Problem: When Kafka Connector uses PEM certificate as user private key for either Kafka broker or schema registry connection, it is possible that job contains warning entries with the message, java.io.IOException: DER input, Integer tag error and multiple Java specific technical information.

Solution: Presence of the earlier mentioned warning entries does not cause job failure nor invalid execution results. Warning message can be ignored or demoted to information level by using message handler.