Multizone Kubernetes and VPC Load Balancer Setup

3 min read

Securely expose your Kubernetes app by setting up a Load Balancer for VPC in a different zone.

You can expose your app to the public by setting up a Kubernetes LoadBalancer service in your IBM Cloud Kubernetes Service cluster. When you expose your app, a Load Balancer for VPC that routes requests to your app is automatically created for you in your VPC outside of your cluster. 

In this post, you will provision an IBM Cloud Kubernetes Service cluster spanning two private subnets (each subnet in a different zone), deploy an application using a container image stored in an IBM Cloud Container Registry and expose the app via a VPC load balancer deployed to a public subnet in a different zone. Sound complex? Don't worry, you will provision and deploy the app using Terraform scripts. 

Here's a simple architecture diagram to show what you will achieve by following the steps in this post:

Here's a simple architecture diagram to show what you will achieve by following the steps in this post:
  1. The administrator provisions everything from VPC to subnets to load balancer using Terraform scripts. 
  2. The user will access the application via the VPC load balancer URL.

Why IBM Cloud Kubernetes Service on VPC?

With IBM Cloud Kubernetes Service clusters on VPC compute, you can create your cluster on VPC infrastructure in the next generation of the IBM Cloud platform, in your Virtual Private Cloud. VPC gives you the security of a private cloud environment with the dynamic scalability of a public cloud. VPC uses the next version of IBM Cloud Kubernetes Service infrastructure providers, with a select group of v2 API, CLI and console functionality. 

Prerequisites

  1. IBM Cloud CLI 
    • IBM Cloud Container Registry CLI plug-in (cr)
    • IBM Cloud Kubernetes Service CLI plug-in (ks)
  2. tfswitch 

Let's get started 

  1. Open a terminal or command prompt on your machine, clone the GitHub repository and move to the directory:
           git clone https://github.com/VidyasagarMSC/vpc-private-k8s-public-lb.git
           cd vpc-private-k8s-public-lb
    The file structure in the GitHub repo should look like this: 
    The file structure in the GitHub repo should look like this: 
  2. Create the local.env file from the template file provided in the repo and update the environment variables accordingly. Once updated, source the file:
          cp templates/local.env.template local.env
          source local.env
  3. Execute the main.sh script that, in turn, runs the Terraform commands and setup.sh shell script:
          ./scripts/main.sh
    • The below are the resources provisioned:
      • A VPC 
      • Three subnets — one in each zone: 
        • Subnets 1 and 2 in zone 1 and zone 2 are private 
        • subnet-lb in zone 3 is public and will be used to deploy the load balancer
      • A public gateway attached to subnet-lb
      • An IBM Cloud Kubernetes Service cluster spanning two zones of subnet 1 and subnet 2
      • A Kubernetes namespace (if provided as part of local.env)
      • A Container Registry namespace (if no registry namespace is mentioned in your local.env)
        lb3
  4. You should see the public load balancer URL in the output. Check the URL with the below cURL command:
        curl <Load_Balancer_URL>:8080

Under the hood

If you are wondering how the load balancer is deployed to a subnet in a different zone, you are in the right section.

First, you need to check the generated file loadbalancer.yaml — specifically, the annotations. If you observe closely, this annotation — service.kubernetes.io/ibm-load-balancer-cloud-provider-vpc-subnets— is doing the magic.

apiVersion: v1
kind: Service
metadata:
 name: myloadbalancer
 namespace: $KUBERNETES_NAMESPACE
 annotations:
   service.kubernetes.io/ibm-load-balancer-cloud-provider-ip-type: public
   service.kubernetes.io/ibm-load-balancer-cloud-provider-vpc-subnets: $SUBNET_ID
spec:
 type: LoadBalancer
 selector:
   app : $MYPROJECT-deployment
 ports:
  - name: http
    protocol: TCP
    port: 8080
    targetPort: 8080

Ensure that you add a label in the metadata section of your deployment configuration file. This custom label identifies all pods where your app runs to include them in the load balancing.

The annotation is to specify one or more subnets to which the VPC application load balancer service deploys. Note that you can specify a different subnet in the same VPC than the subnets to which your cluster is attached. In this case, even though the VPC application load balancer deploys to a different subnet in the same VPC, the VPC application load balancer can still route traffic to your worker nodes on the cluster subnets. To see subnets in all resource groups, run ibmcloud ks subnets --provider vpc-gen2 --vpc-id <vpc> --zone <zone>.

For other supported annotations, check the documentation here. The following provides additional info while using the annotations:

Clean-up

To clean up the setup, run the below command: 

./scripts/cleanup.sh

What's next?

If you have any queries, feel free to reach out to me on Twitter or on LinkedIn

Be the first to hear about news, product updates, and innovation from IBM Cloud