Securing data in motion for Apache Kafka

You must create a secure key and keystore to be able to encrypt and decrypt messages with Apache Kafka.

Procedure

  1. Log on to the computer where Apache Kafka is installed as the root user.
  2. Create a key and a keystore for each Kafka broker. 
    keytool -genkey -alias SIKafkaServerSSL -validity 365 -keystore SIKafkaServerSSLKeystore.jks -dname "CN=si.ibm.com,O=IBM,OU=IBMAnalytics,L=IN,ST=ON,C=CA" -keypass YourKeyPassword
  3. When prompted, enter a password for the key. For example, enter sifs123.
  4. Export the certificate from the keystore. 
    keytool -certreq -file SIKafkaCert -alias SIKafkaServerSSL -keystore SIKafkaServerSSLKeystore.jks
  5. When prompted, enter the password that you used. For example, enter sifs123.
    Note: The certificate must be signed by a certificate authority.
  6. Generate the certificate authority key. 
    openssl req -new -x509 -keyout ca-key -out ca-cert -days 365

    Follow the prompts to generate the key.

  7. Add the key to the server truststore. 
    keytool -import -file ca-cert -keystore SIKafkaServerSSLKeystore.jks -alias CARoot

    The truststore is automatically created.

  8. Add the key to the server keystore. 
    keytool -import -file ca-cert -keystore SIKafkaServerSSLKeystore.jks -alias CARoot
  9. Sign the certificate: 
    openssl x509 -req -CA ca-cert -CAkey ca-key -in SIKafkaCert -out SIKafkaCertSigned -days 365 -CAcreateserial -passin pass:YourPassword
  10. Import the signed certificate into the server keystore: 
    keytool -import -file SIKafkaCertSigned -keystore SIKafkaServerSSLKeystore.jks -alias SIKafkaServerSSL
  11. Update the KafkaInstallLocation/config/server.properties file to include the following text: 
    listeners=SSL://<IP>:<Port>
advertised.listeners=SSL://<IP>:<Port>
ssl.keystore.location=/home/SIUser/SIKafkaServerSSLKeystore.jks
ssl.keystore.password=YourPassword
ssl.key.password= YourKeyPassword
ssl.truststore.location=/home/SIUser/SIKafkaServerSSLTruststore.jks
ssl.truststore.password=YourPassword
ssl.client.auth=required
security.inter.broker.protocol=SSL

    Where <IP> is the IP address where Kafka is running and <Port> can be any open port number, such as 2182.

  12. Copy the SIKafkaServerSSLKeystore.jks and SIKafkaServerSSLTruststore.jks files to the /home/streamsadmin/security directory
    Note: Ensure that the streamsadmin user has access to this file.