Load balancer configuration in a Kubernetes deployment

When deploying API Connect for High Availability, it is recommended that you configure a cluster with at least three nodes and a load balancer. A sample configuration is provided for placing a load balancer in front of your API Connect Kubernetes deployment.

Before you begin

  • This article does not apply to the IBM® DataPower® Gateway component that is a part of the IBM API Connect solution.
  • This article applies to third-party software that IBM does not control. As such, the software may change and this information may become outdated.

About this task

API Connect can be deployed on a single node cluster. In this case the ingress endpoints are host names for which the DNS resolution points to the single IP address of the corresponding node hosting a particular subsystem, and no load balancer is required. For high availability, it is recommended to have at least a three node cluster. With three nodes, the ingress endpoints cannot resolve to a single IP address. A load balancer should be placed in front of an API Connect subsystem to route traffic.

Because it is difficult to add nodes once endpoints are configured, a good practice is to configure a load balancer even for single node deployments. With the load balancer in place, you can easily add nodes when needed. Add the node to the list of servers pointed to by the load balancer and the ingress endpoints defined during installation of API Connect can remain unchanged.

To support Mutual TLS communication between the API Connect subsystems, configure the load balancer with SSL Passthrough and Layer 4 load balancing. In order for Mutual TLS to be performed directly by the API Connect subsystems, the load balancer should leave the packets unmodified, as is accomplished by Layer 4. Following is a description of the communication between the endpoints that are configured with Mutual TLS:

  • API Manager (with the client certificate portal-client) communicates with the Portal Admin endpoint portal-admin (with the server certificate portal-admin-ingress)
  • API Manager (with the client certificate analytics-ingestion-client) communicates with the Analytics Ingestion endpoint analytics-ingestion (with the server certificate analytics-ingestion-ingress)
Note: From v10.0.5.3, it is possible to disable mTLS and use JWT instead, which allows the load-balancers to do TLS termination. For more information, see Enable JWT security instead of mTLS.
Set endpoints to resolve to the load balancer
When configuring a load balancer in front of the API Connect subsystems, the ingress endpoints are set to host names that resolve to a load balancer, rather than to the host name of any specific node. For an overview of endpoints, see Deployment overview for endpoints and certificates.
Use these example topologies as a guideline to determine the best way to configure the load balancer for your deployment.


  • Kubernetes cluster with wildcard DNS

    In this example, API Connect is deployed to a Kubernetes cluster with three master and six worker nodes. The master nodes have the Kubernetes API server listening on port 6443. The Kubernetes API Server is used for communicating with the kubectl command line interface. Note that it is generally not necessary for the Kubernetes API server (on port 6443 in this example) to be exposed outside the cluster through the load balancer, this is just one possible setup shown here.

    The kubernetes/ingress-nginx ingress controller is deployed as a daemonset, so that every worker node in the cluster has an ingress controller pod listening on port 443. The API Connect subsystems (API Manager, Developer Portal, Analytics and Gateway) are all deployed on this same cluster.

    • When you configure a load balancer in front of a Management subsystem, specify timeouts of at least 240 seconds. Note that large deployments might need larger values.

      The default timeout is typically 50 or 60 seconds, which is not long enough to avoid 409 Conflict or 504 Gateway Timeout errors. The 409 Conflict error can occur when the time needed to complete an operation is sufficiently long that a second request gets issued.

      For example, to specify 240 seconds when using HAProxy as a load balancer, set timeout client and timeout server to 240000.

      Best practice is to ensure that the same values are specified for the timeout settings for the load balancer and for the Kubernetes ingress controller. The ingress controller timeout settings are set in the ingress controller config.map. For more information, see the proxy-read-timeout and proxy-send-timeout settings in Kubernetes ingress controller prerequisites .

    • Ensure that the load balancer is configured to support the cipher TLS_ECDHE_RSA_WITH_AES256_GCM_SHA384 (0xc030).

      When the Developer Portal, as a client, calls back to platform-api-endpoint or consumer-api-endpoint of the management subsystem, it sends only a single cipher in the Client Hello. The cipher is TLS_ECDHE_RSA_WITH_AES256_GCM_SHA384 (0xc030). The SSL Handshake will fail if this cipher is not configured (supported) on the client's load balancer.

    A host running HAProxy acts as the load balancer, with a configuration for proxies in the HAProxy configuration file such as:

            log     global
            mode    http
            option  httplog
            option  dontlognull
            timeout connect 5000
            timeout client  240000
            timeout server  240000
            errorfile 400 /etc/haproxy/errors/400.http
            errorfile 403 /etc/haproxy/errors/403.http
            errorfile 408 /etc/haproxy/errors/408.http
            errorfile 500 /etc/haproxy/errors/500.http
            errorfile 502 /etc/haproxy/errors/502.http
            errorfile 503 /etc/haproxy/errors/503.http
            errorfile 504 /etc/haproxy/errors/504.http
    frontend apiservers
            bind *:6443
            mode tcp
            option  tcplog
            option forwardfor
            default_backend k8s_apiservers
    frontend ingress
            bind *:443
            mode tcp
            option  tcplog
            option forwardfor
            default_backend k8s_ingress
    backend k8s_apiservers
            mode tcp
            option  tcplog
            option ssl-hello-chk
            option log-health-checks
            default-server inter 10s fall 2
            server cluster1-master-1 check
            server cluster1-master-2 check
            server cluster1-master-3 check
    backend k8s_ingress
            mode tcp
            option  tcplog
            option ssl-hello-chk
            option log-health-checks
            default-server inter 10s fall 2
            server cluster1-worker-1 check
            server cluster1-worker-2 check
            server cluster1-worker-3 check
            server cluster1-worker-4 check
            server cluster1-worker-5 check
            server cluster1-worker-6 check

    A wildcard DNS record allows the resolution of *.example.com to the IP address of the HAProxy host.

    The ingress endpoints are defined in the CRs (custom resources) for each API Connect subystem. During configuration of each subsystem, you define the endpoint by specifying a host name $STACK_HOST in the CR. For configuration instructions, see Installing the API Connect subsystems.

    You can view the endpoint definitions in sample CRs:

    The following diagram illustrates the wildcard DNS example:
    diagram for load balancer configuration in a Kubernetes deployment
  • Kubernetes cluster without wildcard DNS
    This example uses the same topology as the previous one, except that there is no wildcard DNS resolution. Instead, individual DNS records are added, all pointing to the IP address of the load balancer. For example: portal.example.com and admin.portal.example.com resolve to the IP address of the load balancer.
  • Kubernetes cluster with xip.io or nip.io

    This example uses the same topology as the first one, except that there is no specific DNS resolution configured and instead a service such as xip.io or nip.io provides wildcard DNS resolution. It is not recommended to use this approach for a production setup, as the ingress endpoints configured during installation of API Connect would be tied to the IP address of the load balancer. This approach can be useful when configuring a test deployment in situations where access to DNS records is not practical or would introduce delay in the deployment.