Encrypting cluster data network traffic with IPsec

Encrypt all node-to-node data plane network traffic in your IBM® Cloud Private cluster.

Prerequisites

Every node in your cluster must have at least two network interfaces. One is a management interface and the other interface provides secure networking for the pods. The management network is a separate network that is used by Ansible for the IBM® Cloud Private installation.
Your host file that is used to configure the IBM Cloud Private nodes must contain IP addresses on the management network.

Certain Kubernetes management pods that run on the host network in the IBM Cloud Private cluster uses the management network. The second interface that is used for the pod-to-pod communication is what is secured by IPsec.

Note: Network interface names cannot contain the following strings: docker.*, cbr.*, dummy.*, virbr.*, lxcbr.*, veth.*, lo.*, cali.*, tunl.*, or flannel.*.

Calico networks must be enabled in IP over IP mode.
Ensure that you install libreswan package on all nodes in your cluster that have the Red Hat Enterprise Linux (RHEL) operating system. On nodes that have other operating systems, install the strongswan package. The package is required to enable encryption of data network traffic with IPsec. Ensure that strongswan or libreswan service is configured to start after a node reboots.
All nodes in your cluster must run the same operating system.
To meet Federal Information Processing Standards (FIPS) requirements, follow the guidelines for your operating system:
- For RHEL, see Federal Standards and Regulations
- For Ubuntu, see Certification
- For SUSE Linux Enterprise Server (SLES), see Security Certifications and Verifications

Enable encryption of cluster data network traffic

By default, encryption is disabled in your IBM Cloud Private cluster.

To enable encryption, complete the following tasks:

Verify that the following parameters exist in the config.yaml file. For more information about these parameters, see Network settings:
- network_type: calico
- calico_ipip_mode: Always
- calico_ip_autodetection_method: interface=<data network interface>
- calico_tunnel_mtu: 1390
  
  Set calico_tunnel_mtu to a value that can accommodate the Calico tunnel headers. The Calico tunnel MTU value must be at least 60 bytes less compared to the MTU size of the data network interface that you provided.
Add the following configuration data to the config.yaml file:
```
# IPsec mesh configuration
# If user wants to configure IPsec mesh, the following parameters
# should be configured through config.yaml
ipsec_mesh:
 # To enable IPsec feature
 enable: true
 # List of subnets for which the IPsec should be enabled
 subnets: []
 # List of IPs to be excluded from IPsec subnet
 exclude_ips: []
 # List of ESP encryption/authentication algorithms to be used
 # cipher_suite: "aes128gcm16!"
```
View the following parameter descriptions:
- enable is used to enable or disable encryption of data plane traffic. The default value is false.
- subnets is the network address range of the interfaces that are configured on the cluster nodes for pod-to-pod communication. Subnet address must be specified in CIDR format: [a.b.c.d/n]. If required, list multiple subnets separated by a comma: [a.b.c.d/n, l.m.n.o/p]
- exclude_ips are IP addresses that are excluded from the IPsec subnet. Traffic to these IP addresses is not encrypted. This parameter is optional. Example: [1.1.1.1/32,2.2.2.0/28,3.3.3.3,...]. If you provide the IP address, a netmask of /32 is automatically appended to the IPsec configuration.
- cipher_suite is the Encapsulating Security Payload (ESP) encryption and authentication algorithm to be used. If required, list multiple algorithms separated by a comma. This parameter is optional.
  - The default cipher suite for stongswan ipsec is aes128gcm16!.
  - The default cipher suite for libreswan ipsec is aes_gcm_c128.
  Ensure that this module is available and loaded on all the hosts. You can also change the cipher suite.

Next, continue with IBM Cloud Private installation.

Enabling IPsec mesh after IBM Cloud Private installation

Complete the steps in Enable encryption of cluster data network traffic.

Run the following command:

sudo docker run --net=host -t -e LICENSE=accept -v \
"$(pwd)":/installer/cluster ibmcom/icp-inception:3.1.2 ipsec-mesh

IPsec certificate and key

IPsec implementation in IBM Cloud Private uses Internet Key Exchange (IKE) for mutual authentication between two nodes in your cluster.

For authentication, each node has a digital certificate that is signed by a trusted authority and a private key for that digital certificate.

Node certificates and keys are generated during the installation of IBM Cloud Private.

Using your own certificate and key

You can replace the default certificates and keys after you complete IBM Cloud Private installation. Ensure that you have the following files created:

Certificate authority (CA) file. Example file name: example-ca.crt
Certificate file. Example file name: example-cert.crt
Private key file. Example file name: example-private.key

Replacing certificates and keys on RHEL

Download the OpenSSL package to generate the PKCS #12 bundle file. For more information on the packages in OpenSSL, see OpenSSL Opens in a new tab .

On any RHEL node, generate the PKCS #12 file. Provide your certificate file, private key file, and CA certificate file. Run the following command:
```
 openssl pkcs12 -export -in <example-cert.crt> -inkey <example-private.key> -certfile <example-ca.crt> -out <ipsec-libreswan-example>.p12 -name <ipsec-libreswan-example> -password pass:
```
Note: If you do not want to set a password for the PKCS #12 bundle that you are creating, do not specify a value in the pass: parameter.
Copy the PKCS #12 bundle file that you had generated on to all the other RHEL nodes in your cluster.
On all of your RHEL nodes, import the PKCS #12 bundle file into the Network Security Services database (nssdb). Run the following command, but creating a password is optional. You must use single quotation marks if you do not create a password:
```
 pk12util -i <ipsec-libreswan-example>.p12 -d sql:/etc/ipsec.d -W 'your.password'
```
Replace the certificate name and certificate common name that are in the /etc/ipsec.d/ipsec-libreswan.conf file on all of your RHEL nodes. Your file might resemble the following content:
```
 leftcert=ipsec-libreswan-example
 leftid="CN=ipsec-mesh-example"
 rightid="CN=ipsec-mesh-example"
```
1. To restart the IPsec service on your RHEL nodes, run the following command:
```
service ipsec restart
```

Replacing certificates and keys on other operating systems

Place the CA file in the /etc/ipsec.d/cacerts/ folder.
```
cp example-ca.crt  /etc/ipsec.d/cacerts/
```
Place the certificate file in the /etc/ipsec.d/certs/ folder.
```
cp example-cert.crt  /etc/ipsec.d/certs/
```
Place the private key file in the /etc/ipsec.d/private/ folder.
```
cp example-private.key  /etc/ipsec.d/private/
```
Replace the certificate file name in the /etc/ipsec.conf file. Your input might resemble the following text:
```
leftcert="example-cert.crt"
```
Replace the private key file name in the /etc/ipsec.secrets file with the following key file name:
```
example-private.key
```
Restart the strongSwan service:
```
service strongswan restart
```