Creating SSL artifacts

Before you can enable SSL authentication, you must create SSL certificates, trust stores, and key stores for the Apache Kafka brokers and clients.

About this task

There are two main parts to this procedure:

Creating the key store and trust store for the Kafka brokers (servers).
Creating the key store and trust store for Kafka client applications.

Procedure

Create the key store and trust store for the Kafka brokers (servers)

Ensure that you have OpenSSL 1.0.2 or higher installed and in the path.
To check the version, run the following command:
```
openssl version
```
Generate the self-signed certificate authority (CA) with a command that follows this syntax:
```
openssl req -new -x509 -keyout <ca-key> -out <ca-cert> -days <validity>
```
For example:
```
openssl req -new -x509 -keyout /tmp/ca-key -out /tmp/ca-cert -days 365
```
The generated CA is public-private key pair and certificate that can be used to sign other certificates. You can use the CA key and certificate to sign all of the certificates of the Kafka broker key stores and the Kafka client key stores.

Generate the SSL key store and certificate for the Kafka brokers.

Generate the key and certificate for each machine in the Kafka cluster using the keytool utility. Use the following command syntax:

keytool -keystore <server.keystore.jks> -alias <aliasname> -validity <validity> -genkey -keyalg RSA

For example:

keytool -keystore /tmp/certs/serverkeystore.jks -alias uumdm -validity 365 -genkey -keyalg RSA

When prompted for the common name (CN), provide the fully qualified domain name (FQDN) of the server.
Make note of the key store password and the key password. These will be used later during the configuration.

Sign the certificates in the key store using the CA that you generated in step 2.

Export the certificate from the key store. Use the following command syntax:

keytool -keystore <server.keystore.jks> -alias localhost -certreq -file <cert-file>

For example:

keytool -keystore /tmp/serverkeystore.jks -alias uumdm -certreq -file /tmp/cert-file

If Kafka is running in a Kubernetes pod or a Docker container, the host system has a different host name and IP address compared to the host name and IP address of the pod or container where Kafka is running. Create a SAN entry for the certificate with the host system's host name and IP address.

Create openssl.cnf with the following content. Update the IP.* and DNS.* entries with the details of your deployment's system host names and IP addresses.

default_bits = 2048
default_keyfile = privkey.pem
distinguished_name = req_distinguished_name
req_extensions = req_ext

[ req_distinguished_name ]
countryName = Country
countryName_default = US
stateOrProvinceName = State
stateOrProvinceName_default = California
localityName = City
localityName_default = Santa Cruz
organizationName = Organization
organizationName_default = UC Santa Cruz
commonName = Primary Host Name
commonName_max = 64

[ req_ext ]
subjectAltName = @alt_names

[alt_names]
IP.1 = <IP of the host system where K8 or Docker container is running>
DNS.1 = <Pod host name>
DNS.2 = <IP of the host system where K8 or Docker container is running>

Sign the certificate. Use the following command syntax:

openssl x509 -req -CA <ca-cert> -CAkey <ca-key> -in <cert-file> -out <cert-signed> -days <validity> -CAcreateserial -extfile ./openssl.cnf -extensions req_ext

For example:

openssl x509 -req -CA /tmp/ca-cert -CAkey /tmp/ca-key -in /tmp/cert-file -out /tmp/cert-signed -days 365 -CAcreateserial -extfile ./openssl.cnf -extensions req_ext

Import both the certificate of the CA and the signed certificate into the key store:

Import the CA certificate. Use the following command syntax:

keytool -keystore <serverkeystore.jks> -alias CARoot -import -file <ca-cert>

For example:

keytool -keystore /tmp/serverkeystore.jks -alias CARoot -import -file /tmp/ca-cert

Import the signed certificate. Use the following command syntax:

keytool -keystore <serverkeystore.jks> -alias <aliasname> -import -file <cert-signed>

For example:

keytool -keystore /tmp/serverkeystore.jks -alias uumdm -import -file /tmp/cert-signed

Generate the SSL trust store for the Kafka brokers.
1. Import the CA certificate file created in step 2 to the Kafka server's trust store. Use the following command syntax:
```
keytool -keystore <servertruststore.jks> -alias CARoot -import -file ca-cert -storepass <storepassword>
```
  For example:
```
keytool -keystore /tmp/servertruststore.jks -alias CARoot -import -file ca-cert -storepass xxxxxxx
```
2. Make note of the store password used in this command. This will be used later during the configuration steps.
  
  Note: This newly created trust store holds all of the certificates that the client should trust. Importing a certificate into a trust store means trusting all certificates that are signed by that certificate. This attribute is called the chain of trust. It is particularly useful when deploying SSL on a large Kafka cluster. You can sign all certificates in the cluster with a single CA, and have all machines share the same trust store that trusts the CA. That way, all machines can authenticate all other machines.

Create the key store and trust store for Kafka client applications

If SSL client authentication is enabled (ssl.client.auth=required), then both the trust store and key store files must be created. Without SSL authentication, you only need to create the trust store file.

Create the client trust store file.

Copy the server's <ca-cert> file, then run the following command to create the client trust store file:

keytool -keystore <clienttruststore.jks> -alias CARoot -import -file <ca-cert> -storepass <storepassword>

For example:

keytool -keystore /tmp/clienttruststore.jks -alias CARoot -import -file /tmp/ca-cert -storepass xxxxxxx

Create the client key store file.

Generate the key and certificate for the client using the keytool utility. Use the following command syntax:

keytool -keystore <clientkeystore.jks> -alias <clientlocalhost> -validity <validity> -genkey

For example:

keytool -keystore /tmp/clientcerts/clientkeystore.jks -alias localhost -validity 365 -genkey

Make note of the key store password and the key password. These will be used later during the configuration.

Sign the certificates in the key store using the CA that you generated for the brokers in step 2.

Export the certificate from the key store. Use the following command syntax:

keytool -keystore <clientkeystore.jks> -alias <clientlocalhost> -certreq -file <cert-file>

For example:

keytool -keystore /tmp/clientcerts/clientkeystore.jks -alias localhost -certreq -file /tmp/clientcerts/cert-file

Sign the certificate. Use the following command syntax:

openssl x509 -req -CA <ca-cert> -CAkey <ca-key> -in <cert-file> -out <cert-signed> -days <validity> -CAcreateserial

For example:

openssl x509 -req -CA /tmp/clientcerts/ca-cert -CAkey /tmp/clientcerts/ca-key -in /tmp/clientcerts/cert-file -out /tmp/clientcerts/cert-signed -days 365 -CAcreateserial

Note: The values for <ca-cert> and <ca-key> can be copied from the corresponding values that you used for the brokers.

Import both the certificate of the CA and the signed certificate into the key store.

Import the CA certificate using the following command syntax:

keytool -keystore <clientkeystore.jks> -alias CARoot -import -file <ca-cert>

For example:

keytool -keystore /tmp/clientcerts/clientkeystore.jks -alias CARoot -import -file /tmp/clientcerts/ca-cert

Import the signed certificate using the following command syntax:

keytool -keystore <clientkeystore.jks> -alias localhost -import -file <cert-signed>

For example:

keytool -keystore /tmp/clientcerts/clientkeystore.jks -alias localhost -import -file tmp/clientcerts/cert-signed