Preparing SSL certificates for Apache Kafka

You are given this example to create the self-signed SSL certificates for the Apache Kafka broker server and client. These steps must be done on the system where Apache Kafka is installed.

1. Generate a certificate for each Kafka broker. The default password is password.

   Use the keytool utility, from the jvm package or from Java™ 1.8, to generate the keystore file that stores the certificate.

   
   cd $JAVA_HOME/jre/lib/security
   ../../bin/keytool -keystore server.keystore.jks -alias server_host -validity 365 -genkey -keyalg RSA
   ../../bin/keytool -keystore client.keystore.jks -alias client_host -validity 365 -genkey -keyalg RSA

2. Create your own certificate authority for signing.

   openssl req -new -x509 -keyout ca-key -out ca-cert -days 365

3. Sign all the certificates that are generated in Step 1 with the certificate authority that is generated in Step 2.

   
   ../../bin/keytool -keystore server.keystore.jks -alias server_host -certreq -file cert-file
   ../../bin/keytool -keystore client.keystore.jks -alias client_host -certreq -file cert-file 

4. Import the certificate authority and signed certificate to keystore.

   
   ../../bin/keytool -keystore server.keystore.jks -alias CARoot -import -file ca-cert
   ../../bin/keytool -keystore server.keystore.jks -alias server_host -import -file cert-signed

   The server.keystore.jks file contains ca-cert, cert-signed, and cert-file.

   
   ../../bin/keytool -keystore client.keystore.jks -alias CARoot -import -file ca-cert
   ../../bin/keytool -keystore client.keystore.jks -alias client_host -import -file cert-signed

   The client.keystore.jks file contains ca-cert, cert-signed, and cert-file.

5. Import the certificate authority that you generated into your truststore.

   
   ../../bin/keytool -keystore server.truststore.jks -alias CARoot -import -file ca-cert
   ../../bin/keytool -keystore client.truststore.jks -alias CARoot -import -file ca-cert

6. Add the following section to your server.properties of your Kafka broker server or servers. Then, start your zookeeper and Kafka broker server or servers.

   
   listeners=SSL://server_host:port
   ssl.keystore.location=/opt/ibm-java-8/jre/lib/security/server.keystore.jks
   ssl.keystore.password=yourPassword
   ssl.key.password=yourPassword
   ssl.truststore.location=/opt/ibm-java-8/jre/lib/security/server.truststore.jks
   ssl.truststore.password=yourPassword
   ssl.client.auth=required
   ssl.enabled.protocols=TLSv1.2
   ssl.truststore.type=JKS
   ssl.keystore.type=JKS
   ssl.secure.random.implementation=SHA1PRNG
   ssl.endpoint.identification.algorithm=

7. Configure Kafka clients. Create a client-ssl.properties file with these configuration settings in the $kafka_home/config directory.

   
   security.protocol=SSL
   ssl.truststore.location=/opt/ibm-java-8/jre/lib/security/client.truststore.jks
   ssl.truststore.password=yourPassword
   ssl.keystore.location=/opt/ibm-java-8/jre/lib/security/client.keystore.jks
   ssl.keystore.password=yourPassword
   ssl.key.password=yourPassword
   ssl.enabled.protocols=TLSv1.2
   ssl.truststore.type=JKS
   ssl.keystore.type=JKS
   ssl.endpoint.identification.algorithm=

8. Start a kafka-console-producer and a kafka-console-consumer to verify the communication between the client and server.
   To start a kafka-console-producer:

   $kafka_home/bin/kafka-console-producer.sh --broker-list erver_host:port --topic yourTopic --producer.config $kafka_home/config/client-ssl.properties

   To start a kafka-console-consumer:

   $kafka_home/bin/kafka-console-consumer.sh --bootstrap-server erver_host:port --topic yourTopic --from-beginning --consumer.config $kafka_home/config/client-ssl.properties