Job failures
Jobs with Kafka Connector can abort with error message that requires special steps to be performed.
- Job fails with error message:
SAX parser exception thrown: The input ended before all started tags were ended. Last tag started was 'AdvancedKafkaConfigOptions'- Connector allows to define custom properties specific to Kafka Client. The value of
Usage -> Advanced Kafka Configuration ->
Kafka Client properties can contains multiple lines of key-value entries. As
each configuration property can be parameterized using
#name#convention, this property is also checked if any of parameter is present. At the same time, the value of Kafka Client properties must conform Java Properties class requirements. The#character can be interpreted as either a comment or a start of job parameter. When the#characters appears within the last line of this property, preprocessor assumes that this should be a matching second#character and fails if not found. To avoid the error, make sure that at least new empty line is present at the end of this configuration property.
- On Hortonworks (HDP) 3.0 job fails with error message:
-
com.ascential.e2.common.CC_Exception:org.apache.kafka.common.config.ConfigException:Missing required configuration "partition.assignment.strategy" which has no default value - Each node that is used by the DataStage Engine defines Java classpath that is attached to the
process definition. The classpath contains Kafka Client jar library with an older minimum version
supported by Kafka Connector. Java loads required class definition in order they are defined on
classpath. As system classpath takes precedence over Kafka Connector classpath, the unsupported
library fails to provide required functionality and job fails. On Hortonworks 3.0 platform, the
problematic jar library is
kafka-clients-0.8.2.1.jar. The workaround is to use the minimum supported KafkaClient 0.10.2jar file (kafka-clients-0.10.2.0.jar) and replace the old one with the new content without changing the old name. Make sure to create a backup of original file first.