Resolving problems when using Kafka nodes

Use the advice given here to help you to avoid or resolve problems that might arise when you are using KafkaConsumer or KafkaProducer nodes.

Connecting to a Kafka cluster

Procedure

  • When you set the Kafka user ID and password by using the mqsisetdbparms command, ensure that you used the form kafka::KAFKA::integrationServerName.
  • If you are connecting to IBM® Event Streams in IBM Cloud, ensure that the Security protocol property on the Kafka node is set to SASL_SSL. For more information about configuring the security credentials for connecting to Event Streams, see Using Kafka nodes with IBM Event Streams.

Using log4j 2 to investigate connection and configuration problems

About this task

If you have a problem when you connect to Kafka, and the problem persists after you validate that the configuration of the nodes is correct, you might find more information by enabling tracing from the Kafka client.

The Kafka client uses the log4j 2 component to capture trace information. The lib subdirectory of the IBM App Connect Enterprise installation directory contains a log4j2.properties file, which contains an example of how tracing can be enabled for the Kafka client:

############################################################################
## Kafka logger, un-comment the following section to write Kafka log messages
## to a file.
## The logging level, currently 'DEBUG' is the most detailed level of 
## logging available. The 'log4j.appender.kafkaAppender.Threshold' value may
## be set to one of 
##    ALL, TRACE, DEBUG, INFO, WARN, ERROR, FATAL, and OFF.
## to configure the detail level of the information written to the log file. 
############################################################################
rootLogger.level=OFF

#appender.file.type=File
#appender.file.name=fileLogger
#appender.file.fileName=/var/mqsi/kafka.log
#appender.file.layout.type=PatternLayout
#appender.file.layout.pattern=[%d] %p %m (%c)%n

#rootLogger.appenderRef.file.ref=fileLogger

Procedure

Complete the following steps to enable tracing from the Kafka client:

  1. Copy the log4j2.properties file from the lib subdirectory of the IBM App Connect Enterprise installation directory to the workpath of the integration server:
    For an independent integration server, the target directory is shown in the following example:
    cp <install dir>/server/lib/log4j2.properties <work-dir>/log4j2.properties
    For a managed integration server, the target directory is shown in the following example:
    cp <install dir>/server/lib/log4j2.properties <workpath>/components/<integrationNodeName>/servers/<serverName>/log4j2.properties
    
    For all integration servers that are managed by the same integration node, the target directory is shown in the following example:
    cp <install dir>/server/lib/log4j2.properties <workpath>/components/<integrationNodeName>/log4j2.properties
    Ensure the integration server userid has access and write permissions to the target folder.
  2. In the file that you copied, uncomment the kafkaAppender lines, modify the level of logging and the destination for the captured trace, as required. For example, your new log4j2.properties file might look like this:
    ############################################################################
    ## Kafka logger, un-comment the following section to write Kafka log messages
    ## to a file.
    ## The logging level, currently 'DEBUG' is the most detailed level of 
    ## logging available. The 'log4j.appender.kafkaAppender.Threshold' value may
    ## be set to one of 
    ##    ALL, TRACE, DEBUG, INFO, WARN, ERROR, FATAL, and OFF.
    ## to configure the detail level of the information written to the log file. 
    ############################################################################
    rootLogger.level=ALL
    
    appender.file.type=File
    appender.file.name=fileLogger
    appender.file.fileName=/var/mqsi/kafka.log
    appender.file.layout.type=PatternLayout
    appender.file.layout.pattern=[%d] %p %m (%c)%n
    
    rootLogger.appenderRef.file.ref=fileLogger
  3. Set the environment variable:
    export IBM_JAVA_OPTIONS=-Dbroker.log4j.configuration=/path/to//log4j2.properties
  4. Restart the integration server to start capturing logging information from the Kafka client.
    If no output is logged you can diagnose issues with log4j 2 by setting the java system properties:
    -Dlog4j2.debug -Dorg.apache.logging.log4j.simplelog.StatusLogger.level=trace
    You can combine the log4j 2 debug property with the environment variable from Step 3, remember to separate them with a space in the command. For example:
    export IBM_JAVA_OPTIONS="-Dbroker.log4j.configuration=/path/to//log4j2.properties log4j2.debug -Dorg.apache.logging.log4j.simplelog.StatusLogger.level=trace"
    Restart the integration server or node again so the changes take effect. The log4j 2 debug logging is output to stdout/stderr.
    Note: The default configuration places no limit on the size of the trace file captured, so ensure that you have sufficient capacity in the logging destination for the trace.

Enabling JSSE trace to troubleshoot issues

About this task

If you have a problem when you connect to Kafka and are using an SSL connection, you might find more information by enabling JSSE trace.

Procedure

You can enable JSSE trace by setting the environment variable IBM_JAVA_OPTIONS as follows:

export IBM_JAVA_OPTIONS=-Djavax.net.debug=all

Alternatively, enable JSSE trace by completing the following steps:

  1. Stop the integration node or integration server as described in Starting and stopping an integration node and Stopping an integration server.
  2. In the server.conf.yaml file for the integration server, update the parameter jvmSystemProperty as follows:
    jvmSystemProperty: '-Djavax.net.debug=true'
    For more information about updating the server.conf.yaml file, see Configuring an integration server by modifying the server.conf.yaml file.
  3. Restart the integration node or integration server as described in Starting and stopping an integration node and Starting an integration server.
  4. Re-create the issue.

    If you set the parameter correctly, the text "IBMJSSE2 will not allow protocol SSLv3 per com.ibm.jsse2.disableSSLv3 set to TRUE or default" appears in standard output (STDOUT). If you do not see this text, review your steps and try again. For more information about standard output (STDOUT), see Standard system logs.