Encrypting cluster data network traffic with IPsec

Encrypt all node-to-node data plane network traffic in your IBM® Cloud Private cluster.

Prerequisites

• Every node in your cluster must have at least two network interfaces. One is a management interface and the other interface provides secure networking for the pods. The management network is a separate network that is used by Ansible for the IBM® Cloud Private installation. This network is also used by certain Kubernetes management pods that run on the host network to communicate with other pods on other nodes in the IBM Cloud Private cluster. The second interface that is used for the pod-to-pod communication is what is secured by IPsec.

  Note: Your host file that is used to configure the IBM Cloud Private nodes must contain IP addresses on the management network.

  Note: The network interface names cannot contain the following strings: "docker.*", "cbr.*", "dummy.*", "virbr.*", "lxcbr.*", "veth.*", "lo", "cali.*", "tunl.*", or "flannel.*".

• Calico networks must be enabled in IP over IP mode.

• Ensure that you install libreswan package on all nodes in your cluster that have the Red Hat Enterprise Linux (RHEL) operating system. On nodes that have other operating systems, install the strongswan package. The package is required to enable encryption of data network traffic with IPsec. Ensure that strongswan or libreswan service is configured to start after a node reboots.

  Note: All nodes in your cluster must run the same operating system.

• If you want to make the encryption compliant with Federal Information Processing Standards (FIPS), follow these guidelines:
  • For RHEL, see https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/security_guide/chap-federal_standards_and_regulations
  • For Ubuntu, see https://wiki.ubuntu.com/Security/Certification
  • For SUSE Linux Enterprise Server (SLES), see https://www.suse.com/support/security/certifications/

Enable encryption of cluster data network traffic

By default, encryption is disabled in your IBM Cloud Private cluster.

To enable encryption, complete the following tasks.

1. Ensure that the following parameters exist in the config.yaml file. For more information about these parameters, see Network settings.

   • network_type: calico
   • calico_ipip_enabled: true
   • calico_ip_autodetection_method: interface=<data network interface>

   • calico_tunnel_mtu: 1390

     Note: Set calico_tunnel_mtu to a value such that it can accommodate Calico IP-in-IP (20 bytes) + IPsec header (40 bytes) = 60 bytes. The Calico tunnel MTU value must be at least 60 bytes less compared to the MTU size of the data network interface that you provided.

2. Add the following configuration data to the config.yaml file:

   # IPsec mesh configuration
   # If user wants to configure IPsec mesh, the following parameters
   # should be configured through config.yaml
   ipsec_mesh:
    # To enable IPsec feature
    enable: true
    # The interface for which the IPsec should be enabled.
    interface: <interface name on which IPsec will be enabled>
    # List of subnets for which the IPsec should be enabled
    subnets: []
    # List of IPs to be excluded from IPsec subnet
    exclude_ips: []
    # List of ESP encryption/authentication algorithms to be used
    cipher_suite: aes128gcm16!
   

Following are the parameter descriptions:

• enable is used to enable or disable encryption of data plane traffic. Set to true if you want to enable encryption. Default value: false.
• interface is the IPsec interface. This interface must be the same interface that is set in the calico_ip_autodetection_method parameter.
• subnets is the network address of the interface that is used for your workloads. Subnet address must be specified in CIDR format: [a.b.c.d/n]. If required, list multiple subnets separated by a comma: [a.b.c.d/n, l.m.n.o/p]
• exclude_ips are IP addresses that are excluded from the IPsec subnet. Traffic to these IP addresses is not encrypted. This parameter is optional. Example: [1.1.1.1/32,2.2.2.0/28,3.3.3.3,...].
  Note: When you provide only the IP address, a netmask of /32 is automatically appended to the IPsec configuration.
• cipher_suite is the list of Encapsulating Security Payload (ESP) encryption/authentication algorithms to be used. The default cipher suite that is used is aes128gcm16!.
  Note: Ensure that this module is available and loaded on all the hosts. You can also change it to any cipher suite of your choice.

IPsec certificate and key

IPsec implementation in IBM Cloud Private uses Internet Key Exchange (IKE) for mutual authentication between two nodes in your cluster.

For authentication, each node has a digital certificate that is signed by a trusted authority and a private key for that digital certificate.

These certificates and keys are generated during the installation of IBM Cloud Private.

Using your own certificate and key

You can replace the default certificates and keys with your own certificates and keys after you complete IBM Cloud Private installation. Ensure that you have the following files ready:

• Certificate authority (CA) file. Example file name: example-ca.crt
• Certificate file. Example file name: example-cert.crt
• Private key file. Example file name: example-private.key

On RHEL, run the following commands:

Note: You use the openssl command to generate the PKCS #12 bundle file. The openssl package includes the command. You can download the package from the OpenSSL website.

1. On any RHEL node, generate the PKCS #12 file.

   Provide your certificate file, private key file, and CA certificate file to generate the PKCS #12 bundle file. Run the following command:

    openssl pkcs12 -export -in <example-cert.crt> -inkey <example-private.key> -certfile <example-ca.crt> -out <ipsec-libreswan-example>.p12 -name <ipsec-libreswan-example> -password pass:
   

   Note: If you do not want to set a password for the PKCS #12 bundle that you are creating, do not specify any value in the pass: parameter.

2. Copy the PKCS #12 bundle file that you had generated to all the other RHEL nodes in your cluster.

3. Complete these steps on all the RHEL nodes in your cluster.

   a. Import the PKCS #12 bundle file into the Network Security Services database (nssdb).

    pk12util -i <ipsec-libreswan-example>.p12 -d sql:/etc/ipsec.d -W ''
   

   Note: You must use single quotation marks to specify the password. No password is used in the example. Therefore, no value is provided within the single quotation marks.

   b. Replace the certificate name and certificate common name that are in the /etc/ipsec.d/ipsec-libreswan.conf file.

    leftcert=<name of the `PKCS #12` bundle file>
    leftid=<common name for identification of left endpoint>
    rightid=<common name for identification of right endpoint>
   

   Following is an example code:

    leftcert=ipsec-libreswan-example
    leftid="CN=ipsec-mesh-example"
    rightid="CN=ipsec-mesh-example"
   

   c. Restart the IPsec service:

    service ipsec restart
   

On all operating systems except RHEL, complete the following steps:

1. Place the CA file in the /etc/ipsec.d/cacerts/ folder.
2. Place the certificate file in the /etc/ipsec.d/certs/ folder.
3. Place the private key file in the /etc/ipsec.d/private/ folder.
4. Replace the certificate file name in the /etc/ipsec.conf file: leftcert="example-cert.crt"
5. Replace the private key file name in the /etc/ipsec.secrets file: update the key file name with example-private.key
6. Restart the strongSwan service:

   service strongswan restart