IBM Cloud Private deployment topologies that use Calico

Depending on your environment, you can use Calico for different IBM Cloud Private deployment topologies.

Calico network via specific NICs

Because of consistent network device naming, network interface controllers (NICs) can be named differently for each cluster node, but are physically connected to the same network. For example:

Topology with a single NIC and different names

In this scenario, install IBM Cloud Private by using the following config.yaml configuration:

calico_ip_autodetection_method: interface=ens192,ens168,ens172

Note: If a node has multiple matches in the list, the first one is considered to create the Calico network mesh. This can, however, break the Calico network mesh if unintended interfaces are used. This method is suitable only when a network interface matches only one value in the list.

Separate network for pod-to-pod communication and control plane by using multiple NICs

In environments where multiple networks are available, IBM Cloud Private can be configured to use separate physical networks for control and data planes so that the Kubernetes management traffic is on a separate layer 2 or 3 network segment. In this scenario two networks are configured, 192.168.30.0/24 on eth0 for management traffic, and 172.68.32.0/24 for application traffic.

Multi-nic: separate management/control and data network

In this scenario, install IBM Cloud Private by using the following configuration:

cluster/hosts:

The network that each kubelet communicates with the kube-apiserver on uses the management subnet IP address on 192.168.30.0/24:

[master]
192.168.30.10

[proxy]
192.168.30.10

[worker]
192.168.30.20
192.168.30.21
192.168.30.22

cluster/config.yaml:

Configure the Calico IP mesh either to use the data network interface eth1 or to specify the interface that routes the data network 172.68.32.0/24.

calico_ip_autodetection_method: interface=eth1

Or

calico_ip_autodetection_method: can-reach=<Gateway IP for subnet 172.68.32.0/24>

Workload isolation by using different layer 2 network segments

In multi-tenant scenarios or in scenarios where compliance requires that some workloads need to be isolated from each other, deploy workloads on separate layer 2 segments. Use a router or firewall control and inspect the flow of network traffic between them.

If worker nodes and proxy nodes are already installed on separate layer 2 network segments, IBM Cloud Private can use the podNodeSelector admission controller to deploy pods to a particular namespace target or a particular group of worker nodes. In cases where stateful packet inspection is required for traffic that flows between groups of worker nodes, it is necessary for IP-in-IP encapsulation to be disabled and the router joins the BGP mesh.

For more information on how to configure IBM Cloud Private during installation for isolation, see Isolating network and compute environments.

Pod-to-pod communication encryption with IPsec

Network traffic can be secured if all of the applications that are running inside of pods implement SSL or TLS encryption. This scenario is not always possible, for example when you are running third-party Helm charts that do not support TLS-encrypted connections. To secure pod-to-pod traffic, IBM Cloud Private can be configured to encrypt the pod mesh network by using IPsec in transport mode so that all pod traffic is transparently encrypted. This configuration might be the setup you want in shared untrusted networks like a public cloud.

Enabling the IPsec mesh requires at least two network interfaces; one for management traffic and one for data plane. IPsec also requires IP-in-IP mode to be enabled and adds additional CPU overhead to perform encryption and decryption of pod traffic. For more information, see Encrypting cluster data network traffic with IPsec.

ipsec

cluster/hosts

[master]
192.168.30.10

[proxy]
192.168.30.10

[worker]
192.168.30.20
192.168.30.21
192.168.30.22

cluster/config.yaml

calico_ipip_enabled: true
calico_tunnel_mtu: 1390
calico_ip_autodetection_method: interface=eth1

ipsec_mesh:
  enable: true
  interface: eth1
  subnets: [172.68.32.0/24]
  exclude_ips: [172.68.32.1, 172.68.32.2, 172.68.32.3]
  cipher_suite: aes128gcm16!