Configuring Horizontal Pod Autoscaler

You can configure the parsing service, run service, and runtime service pods to automatically scale horizontally based on workload demand.

About this task

When Horizontal Pod Autoscaler (HPA) is enabled, a new Kubernetes resource called HorizontalPodAutoscaler is created. This resource controls the scale of a deployment and its replica set by adding or removing pods according to the volume of the workload. For more information about HPA, see the Horizontal Pod Autoscaling documentation.

The decision to scale up or down a deployment is based on the comparison of the average CPU usage and average memory consumption of all the pods of the deployment with the configurable targets for both metrics.

Configuration parameters

The following tables list and define the autoscaling parameters:

Table 1. Core autoscaling parameters
Parameter Type Required Default Description
enabled Boolean No
  • true for extra-large profile
  • false otherwise
 Enable or disable autoscaling
min_replicas integer No 2 Minimum number of pod replicas
max_replicas integer No 5 Maximum number of pod replicas
target_cpu_utilization_percentage integer No Auto-calculated. See Table 4, 5, and 6. Target CPU usage as a percentage of request
target_memory_utilization_percentage integer No None Target memory usage as percentage of request
Table 2. Scale-up behavior parameters
Parameter Type Default Description
scaleup.policies_pods_value integer
  • 2 (for the parsing and runtime services)
  • 1 (for the run service)
 Maximum pods to add for each period
scaleup.policies_pods_period_seconds integer 60 How often HPA can apply the scaling action (in seconds)
scaleup.stabilization_window_seconds integer 0 Cool-down window before applying the scale-up action
Table 3. Scale-down behavior parameters
Parameter Type Default Description
scaledown.policies_pods_value integer 1 Maximum pods to remove during each period
scaledown.policies_pods_period_seconds integer
  • 60 (for the parsing and runtime services)
  • 120 (for the run service)
 How often HPA can apply the scaling action (in seconds)
scaledown.stabilization_window_seconds integer 300 Cool-down window before applying the scale-down action (5 minutes)

Understanding target CPU usage

When target_cpu_utilization_percentage is not specified, the operator automatically calculates it based on resources either set explicitly or derived from the profile size.

The calculation uses the formula: (CPU Limit × 80%) / CPU Request × 100%

Table 4. Parsing service target CPU by profile
Profile CPU Request CPU Limit Auto-calculated Target
Small 250 m 1000 m 320%
Medium 500 m 1500 m 240%
Large 1000 m 2000 m 160%
Extra-Large 2000 m 2000 m 80%
Table 5. Run service target CPU by profile
Profile CPU Request CPU Limit Auto-calculated Target
Small 500 m 1500 m 240%
Medium 1000 m 2000 m 160%
Large 1500 m 2000 m 107%
Extra-Large 2000 m 2000 m 80%
Table 6. Runtime service target CPU by profile
Profile CPU Request CPU Limit Auto-calculated Target
Small 500 m 2000 m 320%
Medium 1000 m 2000 m 160%
Large 1500 m 2000 m 107%
Extra-Large 2000 m 2000 m 80%

Procedure

  1. Configure autoscaling for the parsing service.

    The parsing service handles the parsing of decision artifacts in Decision Designer. As parsing is a CPU and memory intensive operation, you can configure autoscaling in the ads_configuration.parsing_service.autoscaling section:

    ads_configuration:
  parsing_service:
    autoscaling:
      enabled: false  # Default: true if designer profile is "extra-large"
      min_replicas: 2
      max_replicas: 5
      target_cpu_utilization_percentage: 320  # Default: auto-calculated (80% of limit as % of request)
      # target_memory_utilization_percentage: <int_value>  # Optional, no default
      scaleup:
        policies_pods_value: 2                    # Max pods to add per interval
        policies_pods_period_seconds: 60          # How often scaling can occur
        stabilization_window_seconds: 0           # Cooldown before scale-up
      scaledown:
        policies_pods_value: 1                    # Max pods to remove per interval
        policies_pods_period_seconds: 60          # How often scaling can occur
        stabilization_window_seconds: 300         # Cooldown before scale-down (5 min)
    Parameter Default value
    enabled
    • true if Decision Designer profile is set to extra-large.
    • false otherwise.
    min_replicas 2
    max_replicas 5
    target_cpu_utilization_percentage Auto-calculated as 80% of CPU limit that is expressed as percentage of CPU request (example: 320% for small profile where the request is 250 m and the limit is 1000 m)
    Scale-up defaults Add up to 2 pods per 60-second interval, no cool down.
    Scale-down defaults Remove up to 1 pod per 60-second interval, 300-second (5 minute) cool-down.
  2. Configure autoscaling for the run service.

    The run service builds and launches decision tests in Decision Designer. As the build is a CPU intensive operation, you can configure autoscaling in the ads_configuration.run_service.autoscaling section:

    ads_configuration:
  run_service:
    autoscaling:
      enabled: false  # Default: true if designer profile is "extra-large"
      min_replicas: 2
      max_replicas: 5
      target_cpu_utilization_percentage: 320  # Default: auto-calculated
      # target_memory_utilization_percentage: <int_value>  # Optional
      scaleup:
        policies_pods_value: 1                    # More conservative than parsing
        policies_pods_period_seconds: 60
        stabilization_window_seconds: 0
      scaledown:
        policies_pods_value: 1
        policies_pods_period_seconds: 60
        stabilization_window_seconds: 300
    Parameter Default value
    enabled
    • true if Decision Designer profile is set to extra-large.
    • false otherwise.
    min_replicas 2
    max_replicas 5
    target_cpu_utilization_percentage Auto-calculated based on profile.
    Scale-up defaults Add up to 1 pod per 60-second interval, no cool-down.
    Scale-down defaults Remove up to 1 pod per 60-second interval, 300-second cool-down.
  3. Configure autoscaling for the runtime service.

    The runtime service executes decision service archives in the decision runtime. The execution can be CPU and memory intensive. You can configure autoscaling in the ads_configuration.decision_runtime_service.autoscaling section:

    ads_configuration:
  decision_runtime_service:
    autoscaling:
      enabled: false  # Default: true if runtime profile is "extra-large"
      min_replicas: 2
      max_replicas: 5
      target_cpu_utilization_percentage: 320  # Default: auto-calculated
      # target_memory_utilization_percentage: <int_value>  # Optional
      scaleup:
        policies_pods_value: 2
        policies_pods_period_seconds: 60
        stabilization_window_seconds: 0
      scaledown:
        policies_pods_value: 1
        policies_pods_period_seconds: 120         # Slower scale-down (2 min)
        stabilization_window_seconds: 300
    Parameter Default value
    enabled
    • true if the runtime profile is set to extra-large.
    • false otherwise.
    min_replicas 2
    max_replicas 5
    target_cpu_utilization_percentage Auto-calculated based on profile.
    Scale-up defaults Add up to 2 pods per 60-second interval, no cool-down.
    Scale-down defaults Remove up to 1 pod per 120-second interval, 300-second cool-down.

Results

After the autoscaling is configured, the operator automatically creates Horizontal Pod Autoscaler resources for the enabled services. You can verify the configuration by using the following command:

oc get hpa -n <namespace>
oc describe hpa <service-name>-hpa -n <namespace>

Example

Example 1: Enabling autoscaling with default settings

ads_configuration:
  parsing_service:
    autoscaling:
      enabled: true
      min_replicas: 2
      max_replicas: 5
      # target_cpu_utilization_percentage will default to 320% (auto-calculated)

Example 2: Configuring custom CPU and memory targets

ads_configuration:
  decision_runtime_service:
    autoscaling:
      enabled: true
      min_replicas: 3
      max_replicas: 10
      target_cpu_utilization_percentage: 250
      target_memory_utilization_percentage: 200

Example 3: Configuring conservative scaling behavior

ads_configuration:
  run_service:
    autoscaling:
      enabled: true
      min_replicas: 2
      max_replicas: 8
      target_cpu_utilization_percentage: 200
      scaleup:
        policies_pods_value: 1          # Add only 1 pod at a time
        policies_pods_period_seconds: 120  # Wait 2 minutes between scale-ups
        stabilization_window_seconds: 60   # 1-minute cooldown
      scaledown:
        policies_pods_value: 1
        policies_pods_period_seconds: 180  # Wait 3 minutes between scale-downs
        stabilization_window_seconds: 600  # 10-minute cooldown

Example 4: Configuring aggressive scaling for high load

ads_configuration:
  parsing_service:
    autoscaling:
      enabled: true
      min_replicas: 5
      max_replicas: 20
      target_cpu_utilization_percentage: 150
      scaleup:
        policies_pods_value: 4          # Add up to 4 pods at once
        policies_pods_period_seconds: 30   # Check every 30 seconds
        stabilization_window_seconds: 0    # No cooldown
      scaledown:
        policies_pods_value: 2
        policies_pods_period_seconds: 120
        stabilization_window_seconds: 300