Changing load balancer timeout settings

To prevent connections from being closed before processes complete, you might need to adjust the timeout settings on your load balancer node.

Installation phase
  • You are not here. Setting up a client workstation
  • You are not here. Setting up a cluster
  • You are not here. Collecting required information
  • You are not here. Preparing to run installs in a restricted network
  • You are not here. Preparing to run installs from a private container registry
  • You are here icon. Preparing the cluster for Cloud Pak for Data
  • You are not here. Preparing to install an instance of Cloud Pak for Data
  • You are not here. Installing an instance of Cloud Pak for Data
  • You are not here. Setting up the Cloud Pak for Data control plane
  • You are not here. Installing solutions and services
Who needs to complete this task?

Cluster administrator A cluster administrator must complete this task.

When do you need to complete this task?
One-time setup You must complete this task if you plan to install any of the following services:
  • Cognos Dashboards
  • Data Gate
  • Data Product Hub
  • Data Virtualization
  • Db2
  • Db2 Warehouse
  • IBM Knowledge Catalog
  • IBM Knowledge Catalog Premium
  • IBM Knowledge Catalog Standard
  • OpenPages
  • Watson Discovery
  • Watson Speech services
  • Watson Studio
  • watsonx.ai
  • watsonx Code Assistant for Z

Even if you don't plan to install the preceding services, you might need to adjust the timeout settings if you are working with large data sets or you have slower network speeds. For example, you might need to increase the timeout value if you receive a timeout or failure when you upload a large file.

About this task

Increasing the load balancer timeout settings prevents connections from being closed before processes complete.

If you plan to install multiple services with different timeout requirements, use the timeout settings for the service with the highest timeout requirement.

Service Timeout value
Cognos Dashboards The minimum recommended timeout is:
  • Client timeout: 600s (10m)
  • Server timeout: 600s (10m)
Data Gate The minimum recommended timeout is:
  • Client timeout: 7500s (125m)
  • Server timeout: 7500s (125m)
Data Product Hub The minimum recommended timeout is:
  • Client timeout: 300s (5m)
  • Server timeout: 300s (5m)
Data virtualization The minimum recommended timeout is:
  • Client timeout: 300s (5m)
  • Server timeout: 300s (5m)
Db2 The minimum recommended timeout is:
  • Client timeout: 300s (5m)
  • Server timeout: 300s (5m)
Db2 Warehouse The minimum recommended timeout is:
  • Client timeout: 300s (5m)
  • Server timeout: 300s (5m)
IBM Knowledge Catalog The minimum recommended timeout is:
  • Client timeout: 300s (5m)
  • Server timeout: 300s (5m)
IBM Knowledge Catalog Premium The minimum recommended timeout is:
  • Client timeout: 300s (5m)
  • Server timeout: 300s (5m)
IBM Knowledge Catalog Standard The minimum recommended timeout is:
  • Client timeout: 300s (5m)
  • Server timeout: 300s (5m)
OpenPages The minimum recommended timeout is:
  • Client timeout: 1800s (30m)
  • Server timeout: 1800s (30m)
Watson Discovery The minimum recommended timeout is:
  • Client timeout: 300s (5m)
  • Server timeout: 300s (5m)
Watson Speech services The amount of time required to process an audio file depends on various factors, such as the length of your audio file and the type of model you use.

For typical use-cases, the recommended timeout is:

  • Client timeout: 1800s (30m)
  • Server timeout: 1800s (30m)

However, you can reduce the timeout if you send audio file in chunks.

Watson Studio The minimum recommended timeout is:
  • Client timeout: 300s (5m)
  • Server timeout: 300s (5m)
watsonx.ai The minimum recommended timeout is:
  • Client timeout: 600s (10m)
  • Server timeout: 600s (10m)
watsonx Code Assistant for Z The minimum recommended timeout is:
  • Client timeout: 600s (10m)
  • Server timeout: 600s (10m)

Procedure

Complete the appropriate task for your environment:

Deployment environment Managed Self-managed
On-premises No changes are required. See Self-managed on-premises deployments
IBM Cloud See IBM Cloud deployments See IBM Cloud deployments
AWS See Red Hat® OpenShift® Service on AWS (ROSA) See Configuring ingress cluster traffic on AWS in the Red Hat OpenShift Container Platform:
Microsoft Azure You cannot change the timeout settings in Azure Red Hat OpenShift (ARO) environments. The default timeout value is 4 minutes (240 seconds). See the Microsoft Azure load balancer documentation.
Important: If you have an internal and external load balancer on your cluster, ensure that you apply the changes to the external load balancer.
Google Cloud Not applicable Not applicable

Self-managed on-premises deployments

The following procedure shows how to change the timeout settings if you are using HAProxy. If you are using a load balancer other than HAProxy, see the documentation for your load balancer for information about how to configure the timeout settings.

If you are using HAProxy, the load balancer node is the OpenShift cluster public node.

  1. On the load balancer node, verify that the HAProxy timeout settings in the /etc/haproxy/haproxy.cfg file meet the minimum requirements.
  2. If the timeout settings is too low, run the following commands to change the timeout values.
    Remember: If you plan to install multiple services with different timeout requirements, use the timeout settings for the service with the highest timeout requirement.
    1. Set the TIMEOUT_SETTING environment variable to the timeout setting to use:
      export TIMEOUT_SETTING=<timeout>

      Ensure that you specify the unit. For example: 300s or 5m.

    2. To increase the timeout client setting, enter the following command:
      sed -i -e "/timeout client/s/ [0-9].*/ ${TIMEOUT_SETTING}/" /etc/haproxy/haproxy.cfg
    3. To increase the timeout server setting, enter the following command:
      sed -i -e "/timeout server/s/ [0-9].*/ ${TIMEOUT_SETTING}/" /etc/haproxy/haproxy.cfg
  3. Run the following command to apply the changes that you made to the HAProxy configuration:
    systemctl restart haproxy


IBM Cloud deployments

The following procedures show how to change the timeout settings if you are using HAProxy. If you are using a load balancer other than HAProxy, see the documentation for your load balancer for information about how to configure the timeout settings.

If you are using HAProxy, the load balancer node is the OpenShift cluster public node.

If you are setting HAProxy timeout settings for Cloud Pak for Data on IBM Cloud, you can configure route timeouts by using the oc annotate command.

  1. Use the following command to set the server-side timeout for the HAProxy route to 360 seconds:
    oc annotate route zen-cpd --overwrite haproxy.router.openshift.io/timeout=360s

    If you don't provide the units, ms is the default.

  2. If your environment also includes IBM Cloud Network Load Balancer for VPC, the IBM Cloud Network Load Balancer for VPC settings override the HA Proxy settings. Use the following command to increase the timeout the IBM Cloud Network Load Balancer for VPC:
    oc patch svc router-default \
    --namespace=openshift-ingress \
    --type=merge \
    --patch '{"metadata": {"annotations": {"service.kubernetes.io/ibm-load-balancer-cloud-provider-vpc-idle-connection-timeout": "360"}}}'
    Important: For watsonx.ai, increase the timeout to 600 seconds:
    oc patch svc router-default \
    --namespace=openshift-ingress \
    --type=merge \
    --patch '{"metadata": {"annotations": {"service.kubernetes.io/ibm-load-balancer-cloud-provider-vpc-idle-connection-timeout": "600"}}}'
  3. Customize other route-specific settings. For more information, Route-specific annotations in the Red Hat OpenShift Container Platform documentation:
Note: On a Virtual Private Cloud (VPC) Gen2 cluster, the load balancer timeout is set to the following values:
  • Server-side connection attempt: 5 seconds
  • Client-side idle connection: 50 seconds
  • Server-side idle connection: 50 seconds

You can change some of these default values. For more information, see Connection timeouts.



Red Hat OpenShift Service on AWS (ROSA) deployments

By default, ROSA uses the AWS Network Load Balancer, which has a fixed timeout limit of 350 seconds.

Most IBM Cloud Pak for Data services can run with the default timeout. However, if you plan to install services that require a higher timeout setting, you must add an AWS Classic Load Balancer to your cluster. For information about configuring the Classic Load Balancer, see Configure the idle connection timeout for your Classic Load Balancer.


What to do next

Now that you've increased the load balancer timeout settings, you're ready to complete Changing the process IDs limit.