Automatically scaling integration server or integration runtime pods in a deployment by using the Horizontal Pod Autoscaler in Red Hat OpenShift

The Horizontal Pod Autoscaler (HPA) is a Kubernetes resource that uses the metrics collected from pods to automatically scale a replication controller or deployment configuration. From your Red Hat® OpenShift® cluster, you can configure autoscaling for your integration server or integration runtime pods by creating a HorizontalPodAutoscaler custom resource that defines a minimum and maximum number of running pods on the deployment, and the CPU utilization that your pods should target.

After you deploy the HorizontalPodAutoscaler object, it queries the CPU utilization metrics for the pods at regular intervals, calculates a utilization ratio, and dynamically adds or removes pods based on the current resource utilization.

Before you begin

Ensure that your cluster administrator has configured cluster metrics.

Ensure that you are logged in to your Red Hat OpenShift cluster.

Note: Horizontal Pod Autoscaling will not work if the custom resource for an integration server or integration runtime does not include a spec.replicas setting.

Procedure

To configure autoscaling for an integration server or integration runtime's pods, complete the following steps for that integration server or integration runtime:

  1. From your local computer, create a file with a .yaml extension (for example, hpafilename_cr.yaml) by copying the following HorizontalPodAutoscaler custom resource, where:
    • metadata.name is a unique name for the HorizontalPodAutoscaler object.
    • metadata.namespace is the namespace where the integration server or integration runtime is deployed.
    • spec.scaleTargetRef.name is set to the name (that is, the metadata.name value) of your integration server or integration runtime.
    • spec.minReplicas and spec.maxReplicas are set to the minimum number (starting from 1) and maximum number (for example, 10) of replica pods.
    • spec.metrics.type.resource.target.averageUtilization is set to the requested average CPU utilization (expressed as a percentage) across all pods; for example, 80.
    Integration server:
    apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: hpaName
  namespace: namespaceName
spec:
  scaleTargetRef:
    kind: IntegrationServer
    name: integrationServerName
    apiVersion: appconnect.ibm.com/v1beta1
  minReplicas: minimumValue
  maxReplicas: maximumValue
  metrics:
  - type: Resource
    resource:
      name: cpu
      target:
        type: Utilization
        averageUtilization: number
    Integration runtime:
    apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: hpaName
  namespace: namespaceName
spec:
  scaleTargetRef:
    kind: IntegrationRuntime
    name: integrationRuntimeName
    apiVersion: appconnect.ibm.com/v1beta1
  minReplicas: minimumValue
  maxReplicas: maximumValue
  metrics:
  - type: Resource
    resource:
      name: cpu
      target:
        type: Utilization
        averageUtilization: number
  2. From the command line, run the following command to deploy a HorizontalPodAutoscaler object. 
    oc apply -f hpafilename_cr.yaml

The HPA will begin to query for metrics on the pods and should begin to increase or decrease the number of replicas to fall within the minimum and maximum values so that the target CPU utilization is maintained across all pods.

For more information about the Horizontal Pod Autoscaler, see Automatically scaling pods with the horizontal pod autoscaler in the Red Hat OpenShift documentation.

Example

The following HorizontalPodAutoscaler custom resource will autoscale pods to fall within the limits of 1 and 10 replicas. When the average CPU utilization across existing pods for the is-toolkit integration server or ir-toolkit integration runtime is greater than 80%, new pods are added to decrease the average utilization to the 80% target. Similarly, when CPU utilization is less than 80%, pods are removed to increase the utilization to the 80% target.

Integration server:
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: toolkit-hpa
  namespace: ace-test
spec:
  scaleTargetRef:
    kind: IntegrationServer
    name: is-toolkit
    apiVersion: appconnect.ibm.com/v1beta1
  minReplicas: 1
  maxReplicas: 10
  metrics:
  - type: Resource
    resource:
      name: cpu
      target:
        type: Utilization
        averageUtilization: 80
Integration runtime:
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: toolkit-hpa
  namespace: ace-test
spec:
  scaleTargetRef:
    kind: IntegrationRuntime
    name: ir-toolkit
    apiVersion: appconnect.ibm.com/v1beta1
  minReplicas: 1
  maxReplicas: 10
  metrics:
  - type: Resource
    resource:
      name: cpu
      target:
        type: Utilization
        averageUtilization: 80

At a suitable interval after the HPA object is deployed, you can check its status by running the following command. (You can alternatively check the status of the HPA object from the YAML view in the Red Hat OpenShift web console.) 

oc get hpa toolkit-hpa

In the updated YAML definition for the HPA object, the spec.status settings will display the current autoscaling status and CPU utilization percentage, as shown in the following example for an integration server.

kind: HorizontalPodAutoscaler
apiVersion: autoscaling/v2
metadata:
  name: toolkit-hpa
  namespace: ace-test
  selfLink: /apis/autoscaling/v2/namespaces/ace-test/horizontalpodautoscalers/toolkit-hpa
  uid: 16cd8a4b-ed16-4f5c-943e-026d4ed24f42
  resourceVersion: '25018016'
  creationTimestamp: '2023-03-17T12:07:20Z'
  annotations:
    autoscaling.alpha.kubernetes.io/conditions: >-
      [{"type":"AbleToScale","status":"True","lastTransitionTime":"2023-03-17T12:07:35Z","reason":"ReadyForNewScale","message":"recommended
      size matches current
      size"},{"type":"ScalingActive","status":"True","lastTransitionTime":"2023-03-17T12:07:36Z","reason":"ValidMetricFound","message":"the
      HPA was able to successfully calculate a replica count from cpu resource
      utilization (percentage of
      request)"},{"type":"ScalingLimited","status":"False","lastTransitionTime":"2023-03-17T12:07:36Z","reason":"DesiredWithinRange","message":"the
      desired count is within the acceptable range"}]
    autoscaling.alpha.kubernetes.io/current-metrics: >-
      [{"type":"Resource","resource":{"name":"cpu","currentAverageUtilization":99,"currentAverageValue":"299m"}}]
spec:
  scaleTargetRef:
    kind: IntegrationServer
    name: is-toolkit
    apiVersion: appconnect.ibm.com/v1beta1
  minReplicas: 1
  maxReplicas: 10
  metrics:
  - type: Resource
    resource:
      name: cpu
      target:
        type: Utilization
        averageUtilization: 80
status:
  observedGeneration: 1
  lastScaleTime: '2023-03-17T12:13:22Z'
  currentReplicas: 2
  desiredReplicas: 2
  currentMetrics:
  - type: Resource
    resource:
      name: cpu
      currentAverageUtilization: 0
      currentAverageValue: 0