Sizing your IBM® Cloud Private cluster

Each IBM Cloud Private cluster has its own characteristics. There are guidelines that provide sample deployment sizes. They have been classified by size and purpose. The considerations are focused on clusters that are either deployed to VMware or OpenStack environments.

Note: The requirements that are listed aren’t minimum requirements.

• Considerations before you size your cluster
• Sample deployments

Considerations before you size your cluster

View the following considerations before you size your cluster.

• Worker nodes (workloads)
• Proxy nodes
• Management nodes
• Large cluster considerations

Worker nodes (workloads)

As you determine the number of worker nodes and the resource configurations, consider the workload that is running.

• If your cluster has a few worker nodes, consider increasing the number of worker nodes while decreasing the size of the nodes for adequate headspace, efficiency, mobility, and resiliency.
• Accommodate the workload mobility.
• Consider the memory that is required for a specific type of workload.
• Consider the memory that is required for other application frameworks.
• The maximum pod per node is 500 and the maximum pod per CPU core is 10.
• The cluster size depends on the worker node number. The pod number depends on the application type and the worker node's configuration.

Proxy nodes

The following considerations are for proxy node sizing. Proxy nodes can be added at any time.

• Consider the total resource sizing versus the number of nodes.
• You must edit your ingress controller to match your workload through your configuration map.
• Your proxy virtual IP address references only a single node at a time.
• (Optional) Consider a load balancer to spread the workload to your proxy nodes with the external IP address.
• Istio also uses the proxy node for running Ingress and Egress gateways.

Management nodes

Larger clusters with more workload require larger management nodes. Management nodes can be added at any time if they were originally externalized.

For proxy nodes, consider the headspace requirements to carry the workload due to a node failure.

Large cluster considerations

Consider the capabilities of IBM Cloud Private when you plan for a large cluster. IBM Cloud Private has extra enterprise capability workloads in addition to Vanilla Kubernetes, adding more services, such as Calico (node to node MeSH). Also, consider monitoring, logging, and the vulnerability assessment.

The following are more considerations:

• The node-to-node mesh starts to fail with 700 nodes in the cluster. You must create a router reflector for BGP Daemons.
• Consider to use etcd outside of your master nodes if you plan on having a cluster with several hundred worker nodes. A separated etcd cluster is ideal to reduce the impact on the master node.
• Be sure to implement load balancing on your master node.
• The number of services in your cluster affects the load on each node. In large clusters with more than 5000 services, you must run your nodes in IP Virtual Server (IPVS) mode.

Note: IPVS takes extra considerations for deployment.

Sample deployments

View the following sample deployments of different sized environments.

• Small IBM Cloud Private environment (resilience medium)
• Medium IBM Cloud Private environment (resilience medium)
• Large IBM Cloud Private environment (resilience high)

Small IBM Cloud Private environment (resilience medium)

To create a testing environment with this particular cluster, you can deploy a single master node and decrease the proxy nodes (resilience low). To provide the most flexibility for your environment, you must not combine node types. The workers included are shaped for Java workloads. See the Worker nodes section.

Medium IBM Cloud Private environment (resilience medium)

To increase your cluster's resilience level, add two extra master nodes.

Large IBM Cloud Private environment (resilience high)

Sizing cluster for 500 worker nodes:

Sizing cluster for 1000 worker nodes:

To increase your cluster resilience level, you must deploy and manage workloads across multiple IBM Cloud Private clusters.