Known issues and workarounds
There are situations when Kafka Connector can behave unexpectedly and when manual user intervention is required. This topic provides guidance for such cases.
Error: Commit cannot be completed
Problem: When Kafka connector is configured to run in Continuous mode, the job might fail with the message:
Kafka Connector was unable to commit messages due to following error: Commit cannot be
completed since the group has already rebalanced and assigned the partitions to another member. This
means that the time between subsequent calls to poll() was longer than the configured
max.poll.interval.ms, which typically implies that the poll loop is spending too much time message
processing. You can address this either by increasing the session timeout or by reducing the
maximum size of batches returned in poll() with max.poll.records.
Solution: Change the connector execution mode to Sequential.
Error: Invalid property value/Usage/MaxMessage
Problem: A job fails with error message: Invalid property value /Usage/MaxMessage. This can happen when you import job that has the Max Messages property that is configured by using job parameter. As the property is removed in recent connector code, DataStage is not able to match the job parameter with current connector configuration properties.
- Open the job and edit Kafka Connector properties.
- Modify any of the stage parameters. It is not required to change particular value. Editing any of the configuration properties trigger stage recompilation.
- Compile a job.
- Run a job. Error must be gone.
Job log with warning: java.io.IOException: DER input, Integer tag error
Problem: When Kafka Connector uses PEM certificate as user private key for either Kafka
broker or schema registry connection, it is possible that job contains warning entries with the
message, java.io.IOException: DER input, Integer tag error
and multiple Java
specific technical information.
Solution: Presence of the earlier mentioned warning entries does not cause job failure nor invalid execution results. Warning message can be ignored or demoted to information level by using message handler.