Building Kafka producers and consumers in C/C++

The Apache Kafka C/C++ client library librdkafka is ported to z/OS. Follow the steps in this section to set up sample producers and consumers in C/C++.

For the C/C++ APIs provided by librdkafka, see librdkafka documentation. The sample C/C++ producer and consumer programs included in Open Enterprise SDK for Apache Kafka are listed in librdkafka examples.

In the following steps, $KAFKA_HOME refers to the installation folder of Open Enterprise SDK for Apache Kafka in the z/OS UNIX region. Replace $KAFKA_HOME with your own installation folder.

Procedure

  1. Set up environment variables with the following steps:
    1. Edit the $KAFKA_HOME/bin/.env file to update the following variables:
      1. KAFKA_HOME: The absolute path of the Kafka installation folder in z/OS UNIX
      2. CC: ibm-clang or ibm-clang64, which is invocation command you use to compile C programs with IBM Open XL C/C++ for z/OS
      3. CXX: ibm-clang++ or ibm-clang++64, which is invocation command you use to compile C++ programs with IBM Open XL C/C++ for z/OS
    2. Execute the .env file to update the environment variables with the command . $KAFKA_HOME/bin/.env.
  2. Build the sample C/C++ programs with the following steps:
    1. Go to the examples folder with the command cd $KAFKA_HOME/examples.
    2. Get your GNU make executable file name, which can be gmake, make, or another name. In the following steps, $GMAKE refers to your own GNU make name.
    3. Clean the folder to remove existing executable files by running this command:
      $GMAKE clean
    4. Compile the sample programs by running the $GMAKE command.
    5. If you want to compile an individual file, run the following command:
      $GMAKE <file name without extension>
      For example, if you want to compile producer.c, run this command:
      $GMAKE producer
  3. Run the sample C/C++ programs with the following steps:
    1. Go to the examples folder with the command cd $KAFKA_HOME/examples.
    2. Run the executable files with the command ./<executable file name> <parameters>. If you type the executable file name, the usage information of the producer is displayed. For example, if you want to see the usage of the executable file producer, type producer and press enter. The usage information displayed is as follows: 
      % Usage: producer <broker> <topic>
      The usage information means that you can execute the producer with two parameters: the Kafka broker and topic name.