Creating Kafka topics

The InfoSphere® MDM deployment includes sample scripts that enable you to create Kafka topics with default partitions and replication factors.

About this task

There are separate topic creation scripts provided for each supported operating system. You can customize the sample scripts to meet the requirements of your implementation.

Procedure

  1. Browse to <MDM_INSTALL_HOME>/KafkaProcessor/bin. Choose the subdirectory that corresponds to your operating system.
  2. Edit createTopics_Sample.sh to update the following values as appropriate for your deployment:
    KAFKA_HOME
    The installation path of the Kafka instance, such as <MDM_INSTALL_HOME>/kafka_<version>.
    KAFKA_HOST_NAME
    The host name of the Kafka server.
    KAFKA_PORT
    The port of the Kafka server. The default value is 9092.
  3. If your deployment uses Kafka 3.0.0 or later, edit the kafka-topics.sh commands in createTopics.sh to replace the --zookeeper argument with --bootstrap-server.
    For example:
    ${KAFKA_BIN}/kafka-topics.sh --create --bootstrap-server ${hostname}:${port} --replication-factor 3 --partitions 1 --topic com-ibm-mdm-statistics-connect-offsets
    Note: This change is required because the way topics are configured has changed in Kafka 3.0.0. For more information, see the list of notable changes in Kafka 3.0.0 in the Apache Kafka documentation.
  4. Run createTopics.sh to create the Kafka topics for your InfoSphere MDM deployment.