Load Balancing API Calls Across Regions with IBM Cloud Internet Services and Cloud API Gateway

By Matt Hamann and David Green

IBM Cloud API Gateway and IBM Cloud Internet Services: Load balancing traffic across two geographically-separated backends built on IBM Cloud Functions

Redundancy is a key aspect of engineering reliable cloud systems and applications. It’s often desirable to deploy services across regions in addition to multiple data centers. IBM Cloud provides the tools to enable intelligent, geographically-distributed load balancing regardless of backend infrastructure.

In this article, we’ll explore load balancing traffic across two geographically-separated backends built on IBM Cloud Functions. We’ll use the IBM Cloud API Gateway to deploy the same API definition in both regions, and then intelligently route traffic with IBM Cloud Internet Services.

Prerequisites

Before you begin, you’ll need the following:

  • An IBM Cloud account

  • A registered domain name for which you control the DNS

Table of contents

Section 1: Create a Cloud Functions API in region 1

Section 2: Create a Cloud Functions API in region 2

Section 3: Provision Cloud Internet Services (CIS), attach your domain, and generate Origin Certificates

Section 4: Provision Certificate Manager & import Origin Certificates

Section 5: Configure DNS records in CIS

Section 6: Attach registered domain name to API endpoints

Section 7: Create a global load balancer (GLB) in CIS to balance traffic between both endpoints

Section 1: Create a Cloud Functions API region 1

  1. Navigate to the IBM Cloud Functions page. We’ll use the Dallas region in this article.

  2. Click  Start Creating > Create Action.
    Create Action

  3. Name your action with the Default Package and Node.js 10 Runtime selected before creating the action.
    Create Action

  4. Hardcode the action to return the region it was created in. Save and invoke to test the action.
    function main(params) {
      return { message: 'Hello from Dallas’ };
    }
    Hardcode

  5. Now, let’s create an API by navigating to the APIs section in the sidebar.
    APIs

  6. Name your API and enter a base path `/api`. Next, click Create operation.
    Create operation

  7. In the create operation dialog, enter a Path (e.g., “/hello”) and ensure the Action we created previously is selected. Then, click Create.
    Create

  8. At the bottom of the page, click Save & expose.
    Save & expose

  9. The API summary panel should appear. The base URL of the API is displayed near the top.
    API

If we invoke this API, we will get back the message hardcoded into the function (e.g., “Hello from Dallas”).

Note: You will need to append the relative path to the end of the managed url. In this case, it is ‘/hello’

curl -X GET https://9f061a0e.us-south.apiconnect.appdomain.cloud/api/hello
{
  "message": "Hello from Dallas"
}

Section 2: Create a Cloud Functions API in region 2

Let’s create an identical API in another region.

  1. Navigate back to the IBM Cloud Functions page and change to a different region (e.g., London).
    IBM Cloud Functions

  2. Repeat the process from Section 1. Name your action with the Default Package and Node.js 10 Runtime, just like the previous action.
    Default Package

  3. Hardcode the action to return the region in which it is created. Save, then Invoke to test the action.
    function main(params) {
      return { message: 'Hello from London' };
    }
    Save

Again, we need to enable this action to be invoked via an API.

  1. Navigate back to the APIs section, then click Create a Cloud Functions API.

  2. Name your API and enter the same Base path as the other API (e.g., “/api“). As before, click Create operation.
    Create operation

  3. Enter the same Path as the first API (i.e., “/hello”) and select the Action we created in this region, then click Create.
    Create

  4. Once again, click Save & expose at the bottom of the page and the Summary view should appear.
    Save & expose

    If we invoke this API with the relative path added, it will return the region in which it was created.
    curl -X GET https://e60ce1f4.eu-gb.apiconnect.appdomain.cloud/api/hello
    {
      "message": "Hello from London"
    }

Section 3: Provision Cloud Internet Services

So far, we can invoke each API via a URL; however, the domain name is region specific. If we’re going to globally balance traffic between them, we’ll want a common custom domain name like `glb.mycompany.com`. Let’s use Cloud Internet Services (CIS) to set this up.

Note: You’ll need a registered domain name and access to DNS or nameserver settings in order to configure CIS.

With custom domains, we can get a much cleaner URL on which to invoke our serverless API. However, in order to use a custom domain, we will need to upload a SSL certificate to the Certificate Management service. We can conveniently order a SSL certificate from the Cloud Internet Service (CIS).

  1. To provision CIS, search the IBM Cloud catalog for Cloud Internet Services, then click on the entry in the catalog.
    Cloud Internet Services

  2. From the service detail page, select the desired plan and then click Create.
 The Free Trial plan will work for the purposes of this tutorial.
    Free Trial

  3. Begin configuring your domain by clicking the Let’s get started button.
    Let’s get started

  4. Add your domain, then click Connect and continue.
    Connect and continue

In your domain registrar’s control panel, you’ll need to configure your domain’s nameservers to point to CIS. Click Next Step to continue following the CIS setup wizard to complete the nameserver configuration. Once that’s done, the domain status should change from pending (grey) to active (green)
.

Next Step

Mode

Next, we need to order a free “origin” SSL certificate for our domain. Origin certificates are signed by CIS in order to establish a trusted TLS connection between CIS and a backend server. In this case, the backend server will be the IBM Cloud API Gateway.

By the way, if you’re wondering about certificate requirements for your public facing domain (e.g., glb.mycompany.com), don’t worry. CIS will handle all of that for you when we set up a global load balancer later in this tutorial.

  1. Navigate to the Security > Origin page and click Order certificate.
    Order certificate

  2. In the dialog box that appears, enter your domain name, plus a wildcard, such as *.mycompany.com.

  3. Set the Certificate expiration time for the desired length. Since origin certificates aren’t public-facing, we recommend a longer duration (e.g., 15 years).

  4. Click Order.
    Order

  5. In the resulting dialog, copy the Origin certificate and Private key values and save them each to separate files (with extension .pem) for use in the next step.
    Origin certificate

Section 4: Provision Certificate Manager and import origin certificate

Now that we have an origin certificate for our domain, we need to provision the Certificate Manager service to store our certificate and private key for use by the IBM Cloud API Gateway. Certificate Manager is integrated with a wide variety of IBM Cloud services in order to enable centralized storage and management of SSL/TLS certificates.

  1. Navigate to the IBM Cloud catalog and search for Certificate Manager, then click on the resulting entry. Once the service detail page loads, click Create.
    IBM Cloud catalog

  2. Certificate Manager will provision. Once it finishes launching, click Import Certificate on the right-hand side.
    Import Certificate

  3. Enter a Name for the certificate.

  4. Upload the Origin certificate and Private key obtained in Section 3 by clicking Browse next to the Certificate file and Private key file fields respectively. (Note: In this use case, the private key file is required even though Certificate Manager doesn’t require it).

  5. Click Import at the bottom of the page.
    Import

    You should now see an entry for your uploaded certificate
.
    uploaded certificate

Section 5: Configure DNS records in CIS

With the certificate in place, it is almost time to configure our custom domain. Begin by navigating to the Custom Domains tab in the APIs section of IBM Cloud.

Custom Domains

APIs

  1. From the Custom Domains page, use the Region selector to select the two regions in which we created Cloud Functions APIs as part of Sections 1 and 2.
    Region

  2. Look for the Cloud Functions heading and find the entries matching the two APIs created in Sections 1 and 2.

  3. Copy the Default domain/alias for both domains.
    Default domain/alias

    Since we want to access these API endpoints using our own domain name (e.g., glb.mycompany.com), we need to prove that we own the target domain. This is achieved by creating TXT records in our DNS configuration, which we’ll do in CIS below.

  4. Return to Cloud Internet Services, select the Reliability > DNS section, and scroll to the bottom of the page to find the DNS Records section.
    DNS Records

  5. To add a TXT record, click the dropdown menu for the Type and select TXT.

  6. In the Name field, let’s specify a subdomain, such as “glb”, and add one of the default domains to the Content field.

  7. To ensure a quick update, set the TTL to 2 minutes.

  8. Click Add record.
    Add record

  9. Repeat the steps above to add the second domain alias as a TXT record.
    record

  10. After both TXT records are added, the DNS table should look something like this:
    DNS table

Section 6: Attach registered domain name to API endpoints

It’s time to associate the custom domain and origin certificate with the API endpoints.

  1. Navigate back to the Custom Domains tab in the IBM Cloud APIs console.

  2. Click on one of the default domains to open the custom domain settings dialog.

  3. Tick the Assign custom domain checkbox.

  4. Enter the desired Domain name (e.g., glb.mycompany.com).

  5. Select the Certificate manager service instance that was provisioned in Section 4. (It may already be selected by default.)

  6. Open the Certificate dropdown and select the certificate imported as part of Section 4.

  7. Finally, click Save.
    Save

Once again, repeat the same steps for the other default domain, making sure to use the same domain name (e.g., glb.mycompany.com), Certificate Manager service instance, and certificate.

Certificate Manager

Afterward, the Custom Domains page should look something like the following figure. Note the custom domain column now displays the domain name that we plan to use when calling our multi-region API.

Custom Domains

Section 7: Create a global load balancer (GLB) in CIS to balance traffic between both endpoint

The last step before we can test the flow end-to-end is to configure a global load balancer.

  1. Return to the Cloud Internet Services instance created in Section 3 and navigate to Reliability > Global Load Balancers in the left-nav. Then, click Create load balancer.
    Create load balancer

On the global load balancer page:

  1. Set the Balancer hostname to the same subdomain used in Step 5 to create the custom domain (e.g., glb.mycompany.com).

  2. Enable the proxy setting.

  3. Click Add pool.
    Add pool

The origin pool is made up of a set of endpoints that can be proxied by the load balancer. We’ll use this as a way to enable resilience in the event one region fails. Additionally, we can later create rules that route traffic to the closest region based on geography. In this case, we’re going to create two origin pools—one per region.

  1. Provide a Name for the origin pool (e.g., “API_Backends”).

  2. Create a new Health check, setting the protocol to HTTPS and the path to `/_gwhealth`. Leave the port blank to use the default. (Note: The /_gwhealth path is a special health check provided by the gateway for every configured endpoint.)

  3. Set the Health check region to the closest region to endpoint 1, likely Eastern North America.

  4. In the Origins field, give it a human-friendly name (e.g., Serverless), then input the “domain alias” from the first region in the APIs console’s custom domains page.

  5. Click Add to create the origin pool.
    Add to create the origin pool

  6. Repeat the steps above for our second region, setting the Health check region appropriately (e.g., Western Europe) and using the domain alias from the second region in the origin field.
    Health check region

  7. Verify that the load balancer contains two origin pools, then click Provision 1 Resource to create the load balancer.
    Provision 1 Resource

It will take a few moments for the Health status to change, and you will need to refresh the page to see the status change. Once the endpoint health is verified, the load balancer and origins should show as healthy, looking similar to this
:

Health status

Since the load balancer will hit the highest priority pool first and fail-over into the next highest priority pool, when we issue several curl commands to the endpoint, it will return the same response unless one origin becomes unhealthy and it fails over into the second pool.

$ curl -X GET https://glb.mycompany.com/api/hello
{
  "message": "Hello from Dallas"
}

$ curl -X GET https://glb.mycompany.com/api/hello
{
  "message": "Hello from London"
}

Bonus: Enable intelligent routing to the closest region

We can take this solution a step further by enabling intelligent traffic routing to the region nearest the API client. This will change the load balancer to hit the region that is closest to the caller and fail-over into other regions if the closest is unavailable. We can configure these rules by editing the load balancer’s geo route section.

  1. From the load balancer’s summary page, scroll to the bottom and expand the Configure geo routes (optional) section.

  2. Click Add route.
    Add route

  3. Select the region from which the desired traffic originates (e.g., “Eastern North America”) and then click Add.
    Add

  4. Next, associate a pool with the region by clicking Add pool.
    Add pool

  5. Select the desired origin pool (e.g., “API_Backends-Dallas”), then click Add.
    origin pool

  6. Repeat the above steps to create routes for other regions (e.g., “Western Europe”) and then add origin pool targets to those routes (e.g., “API_Backends-London”). Depending on your current location, you’ll want to create a geo route closest to you and associate a specific origin pool so you can see the effects of the routing rules.

When complete, the config will look something like this. If everything looks good, click Apply changes to put the routing rules into effect.

Apply changes

Now, the nearest API endpoint will receive traffic regardless of the pool priority. This reduces the latency of the API request. If the nearest endpoint is unhealthy, the load balancer will fall back to the origin with the highest priority that is healthy.

Conclusion

This article was focused mainly on the “how-to” of load balancing API calls across geographic regions. The simple “hello world” Cloud Functions API can be replaced with a more powerful application code. In addition, the global load balancer can easily handle more than two origins.

There are many additional complexities to running an application in a geographically distributed manner that are beyond the scope of this guide. For more detail on topics like data replication, see the IBM Cloud docs.

As always, we’d love to hear your feedback and questions! Get help for technical questions at Stack Overflowusing the following tags:

  • Cloud Functions: ibm-cloud-functions

  • API Gateway: ibm-cloud & api-management

  • Internet Services: ibm-cloud & cis

For defect or support needs, use the Support section in the IBM Cloud menu.

To get started with any of the IBM Cloud services mentioned here, check them out in the IBM Cloud Catalog.

Be the first to hear about news, product updates, and innovation from IBM Cloud