Starting with Kubernetes version 1.15 in IBM Cloud Kubernetes Service, you can now specify custom pod and service subnets.
Prior to this change, when you created a cluster, only the following default IP ranges were assigned to pods and services within the cluster:
- Pod Subnet: 172.30.0.0/16
- Service Subnet: 172.21.0.0/16
If you would like to customize these IP ranges, you can now specify CIDRs for custom subnets when you create a Kubernetes version 1.15 or later cluster.
NOTE: This cannot be changed later so you must choose your custom subnets wisely.
Customers of the IBM Cloud Kubernetes Service have requested this feature to help them better manage their clusters' subnets while also maintaining connections to their on-premises networks via VPN or Direct Link. Being able to deviate from the default pod and service subnet range can help avoid subnet range overlaps.
For example, if you set up a VPN connection between your cluster and your on-prem data center, you might have conflicts between the default 172.30.0.0/16 range for pods and 172.21.0.0/16 range for services in your cluster and these ranges in your on-prem subnets.
Specifying custom subnets during cluster creation
To specify custom subnets, you must create a Kubernetes version 1.15 or later cluster using the IBM Cloud Kubernetes Service CLI plug-in.
Cluster create command flags
--pod-subnet: Specify a custom subnet CIDR to provide private IP addresses for pods. When you choose a subnet size, consider the size of the cluster that you plan to create and the number of worker nodes that you might add in the future. A subnet that is
/23 will limit cluster to roughly 4 to 8 worker nodes depending on the number of pods on each node. See the documentation for additional details.
--service-subnet: Specify a custom subnet CIDR to provide private IP addresses for services. The subnet must be at least
/24, which allows a maximum of 255 services in the cluster.
Note: The subnets that you specify in these flags cannot be in the following reserved ranges:
- 10.*.*.*: Reserved for IBM Cloud Kubernetes Service workers
- 172.20.*.*: Reserved for internal use
- 192.168.255.*: Reserved for internal use
Cluster create command example
This example command creates a version 1.15 cluster with two worker nodes in the
dal10 zone. Enough pods for up to 32 worker nodes can be created in the 192.168.64.0/20 IP range, and up to 1023 services can be created in the 192.168.128.0/22 IP range.
ibmcloud ks cluster create classic --name custom-subnet --pod-subnet 192.168.64.0/20 --service-subnet 192.168.128.0/22 --workers 2 --machine-type u2c.2x4 --private-vlan 2565679 --public-vlan 2608161 --location dal10 --kube-version 1.15
Pod output example
Pods that do not need host networking will appear within your specified subnet range. For example, the CoreDNS pod below shows up as 192.168.77.199, which falls inside the specified 192.168.64.0/20 range specified during cluster create.
kubectl get pods -n kube-system NAME READY STATUS RESTARTS AGE coredns-6d59786485-68wqs 1/1 Running 0 103m 192.168.77.199 10.95.109.110 <none> <none> coredns-autoscaler-64f9c5b4df-qb5fv 1/1 Running 0 103m 192.168.77.197 10.95.109.110 <none> <none> ibm-file-plugin-76bcf7b78c-4g8kw 1/1 Running 0 101m 192.168.77.198 10.95.109.110 <none> <none> ibm-storage-watcher-689c5d494c-gtlfr 1/1 Running 0 101m 192.168.77.194 10.95.109.110 <none> <none> kubernetes-dashboard-7996b848f4-jn42d 1/1 Running 0 100m 192.168.77.193 10.95.109.110 <none> <none> metrics-server-8c88b5967-55zhw 2/2 Running 0 96m 192.168.77.200 10.95.109.110 <none> <none> public-crbl0s9vs20s1ug7jdipe0-alb1-69789c849f-8bflf 4/4 Running 0 88m 192.168.77.202 10.95.109.110 <none> <none> vpn-788669485d-br4n9 1/1 Running 0 100m 192.168.77.195 10.95.109.110 <none> <none>
Service output example
Services will also appear with your specified subnet range.
kubectl get svc -n kube-system NAMESPACE NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) kube-dns ClusterIP 192.168.128.10 <none> 53/UDP,53/TCP 106m kubernetes-dashboard ClusterIP 192.168.131.194 <none> 443/TCP 104m metrics-server ClusterIP 192.168.131.54 <none> 443/TCP 104m public-crbl0s9vs20s1ug7jdipe0-alb1 LoadBalancer 192.168.131.231 22.214.171.124 80:30456/TCP,443:30531/TCP 92m
More information can be found in the IBM Cloud Kubernetes Service documentation.