Automatically scaling resources for services

Some IBM Cloud Pak for Data services support autoscaling by using the Red Hat® OpenShift® Horizontal Pod Autoscaler (HPA). The HPA changes the resource allocation of services by increasing or decreasing the number of pods in response to CPU or memory consumption.

Permissions that you need for this task
You must be either:
  • A cluster administrator
  • An instance administrator
When you need to complete this task
When you want to dynamically scale resources for services based on usage.

Instead of using the manual, fixed approach to scaling resources for services, you can enable autoscaling to dynamically scale resources up and down based on the user workloads. Autoscaling helps ensure that clusters are not underutilized while workloads that require more resources cannot use the available resources.

The following table shows the services that support autoscaling. You can see detailed instructions for each service in the procedure for automatically scaling resources for services.

Service name Supports the autoScaleConfig setting to automatically scale resources
Cloud Pak for Data control plane ✓
Canvas ✓
Common core services ✓
IBM Cloud Pak foundational services Not supported
AI Factsheets ✓
Analytics Engine powered by Apache Spark Not supported
Cognos Analytics ✓
Cognos Dashboards ✓
Data Gate Not supported
Data Privacy ✓
Data Product Hub ✓
Data Refinery ✓
Data Replication Not supported
DataStage ✓
Data Virtualization Supports autoscaling. Does not use autoScaleConfig.
Db2 Not supported
Db2 Big SQL Not supported
Db2 Data Management Console ✓
Db2 Warehouse Not supported
Decision Optimization ✓
EDB Postgres Not supported
Execution Engine for Apache Hadoop Not supported
IBM Knowledge Catalog (all editions) Not supported
IBM Match 360 with Watson ✓
Informix Not supported
MANTA Automated Data Lineage Not supported
MongoDB Not supported
OpenPages ✓
Orchestration Pipelines ✓
Planning Analytics ✓
Product Master ✓
Scheduling service Not supported
SPSS Modeler ✓
Synthetic Data Generator See Canvas.
Voice Gateway Not supported
Watson Discovery Not supported
Watson Machine Learning ✓
Watson Machine Learning Accelerator ✓
Watson OpenScale ✓
Watson Speech services Not supported
Watson Studio ✓
Watson Studio Runtimes Not supported
watsonx.ai ✓
watsonx Assistant ✓
watsonx Code Assistant for Red Hat Ansible® Lightspeed ✓
watsonx Code Assistant for Z ✓
watsonx Code Assistant for Z Code Explanation Not supported
watsonx.data ✓
watsonx.governance ✓
watsonx Orchestrate Not supported

Before you begin

Best practice: You can run the commands in this task exactly as written if you set up environment variables. For instructions, see Setting up installation environment variables.

Ensure that you source the environment variables before you run the commands in this task.

Procedure

  1. Log the cpd-cli in to the Red Hat OpenShift Container Platform cluster: 
    ${CPDM_OC_LOGIN}
  2. Enable or disable autoscaling for services as needed:
    Cloud Pak for Data control plane

    • Run the following command to enable horizontal autoscaling for the service:

      oc patch ZenService lite-cr \
--namespace ${PROJECT_CPD_INST_OPERANDS} \
--type=merge \
--patch '{"spec": {"autoScaleConfig": true}}'

    • Run the following command to disable horizontal autoscaling for the service:

      oc patch ZenService lite-cr \
--namespace ${PROJECT_CPD_INST_OPERANDS} \
--type=merge \
--patch '{"spec": {"autoScaleConfig": false}}'
    Canvas

    • Run the following command to enable horizontal autoscaling for the service:

      oc patch canvasbase canvasbase-cr \
--namespace ${PROJECT_CPD_INST_OPERANDS} \
--patch '{"spec":{"autoScaleConfig":true}}' \
--type=merge

    • Run the following command to disable horizontal autoscaling for the service:

      oc patch canvasbase canvasbase-cr \
--namespace ${PROJECT_CPD_INST_OPERANDS} \
--patch '{"spec":{"autoScaleConfig":false}}' \
--type=merge
      Canvas is a dependency for the following services:
      • SPSS Modeler. If you want to scale SPSS Modeler, you must scale Canvas as well.
      • Synthetic Data Generator. If you want to scale Synthetic Data Generator, you must scale Canvas.
    Common core services

    • Run the following command to enable horizontal autoscaling for the service:

      oc patch ccs ccs-cr \
--namespace ${PROJECT_CPD_INST_OPERANDS} \
--patch '{"spec":{"autoScaleConfig":true}}' \
--type=merge

    • Run the following command to disable horizontal autoscaling for the service:

      oc patch ccs ccs-cr \
--namespace ${PROJECT_CPD_INST_OPERANDS} \
--patch '{"spec":{"autoScaleConfig":false}}' \
--type=merge
    AI Factsheets

    • Run the following command to enable horizontal autoscaling for the service:

      cpd-cli manage update-cr \
--component=factsheet \
--cpd_instance_ns=${PROJECT_CPD_INST_OPERANDS} \
--patch='{\"autoScaleConfig\":\"true\"}'

    • Run the following command to disable horizontal autoscaling for the service:

      cpd-cli manage update-cr \
--component=factsheet \
--cpd_instance_ns=${PROJECT_CPD_INST_OPERANDS} \
--patch='{\"autoScaleConfig\":\"false\"}'
    Cognos Analytics
    The autoScaleConfig setting is applied to the dynamic CR. To find the name of the CR, run the following command: 
    cr_name = $(oc -n ${PROJECT_CPD_INST_OPERANDS} get caserviceinstance --no-headers -o custom-columns=NAME:.metadata.name)

    • Run the following command to enable horizontal autoscaling for the service:

      oc patch caserviceinstance ${cr_name} 
--type merge 
 -p "{\"spec\":{\"autoScaleConfig\":\"true\"}}" 
 -n ${PROJECT_CPD_INST_OPERANDS}

    • Run the following command to disable horizontal autoscaling for the service:

      oc patch caserviceinstance ${cr_name} 
--type merge 
 -p "{\"spec\":{\"autoScaleConfig\":\"false\"}}" 
 -n ${PROJECT_CPD_INST_OPERANDS}
    Note: If Cognos Analytics is installed in a tethered project, use -n ${PROJECT_CPD_INSTANCE_TETHERED} in the command.
    Cognos Dashboards

    • Run the following command to enable horizontal autoscaling for the service:

      cpd-cli manage update-cr \
--component=dashboard \
--cpd_instance_ns=${PROJECT_CPD_INST_OPERANDS} \
--patch='{\"autoScaleConfig\":true}'

    • Run the following command to disable horizontal autoscaling for the service:

      cpd-cli manage update-cr \
--component=dashboard \
--cpd_instance_ns=${PROJECT_CPD_INST_OPERANDS} \
--patch='{\"autoScaleConfig\":false}'
    Data Privacy

    • Run the following command to enable horizontal autoscaling for the service:

      cpd-cli manage update-cr \
--component=dp \
--cpd_instance_ns=${PROJECT_CPD_INST_OPERANDS} \
--patch='{\"autoScaleConfig\":\"true\"}'

    • Run the following command to disable horizontal autoscaling for the service:

      cpd-cli manage update-cr \
--component=dp \
--cpd_instance_ns=${PROJECT_CPD_INST_OPERANDS} \
--patch='{\"autoScaleConfig\":\"false\"}'
    Data Product Hub5.0.1 or later

    • Run the following command to enable horizontal autoscaling for the service:

      cpd-cli manage update-cr \
--component=dataproduct \
--cpd_instance_ns=${PROJECT_CPD_INST_OPERANDS} \
--patch='{\"autoScaleConfig\":\"true\"}'

    • Run the following command to disable horizontal autoscaling for the service:

      cpd-cli manage update-cr \
--component=dataproduct \
--cpd_instance_ns=${PROJECT_CPD_INST_OPERANDS} \
--patch='{\"autoScaleConfig\":\"false\"}'
    Data Refinery
    • Run one of the following commands to enable horizontal autoscaling for the service:
      • If your installation uses the CR name datarefinery-cr (for example, if this is a new installation of Data Refinery on Cloud Pak for Data version 5.0.x):
        oc patch  DataRefinery datarefinery-cr \
--namespace ${PROJECT_CPD_INST_OPERANDS} \
--patch '{"spec":{"autoScaleConfig":true}}' \
--type=merge
      • If your installation uses the CR name datarefinery-sample (for example, if you upgraded to Data Refinery on Cloud Pak for Data version 5.0.x from an earlier version):
        oc patch  DataRefinery datarefinery-sample \
--namespace ${PROJECT_CPD_INST_OPERANDS} \
--patch '{"spec":{"autoScaleConfig":true}}' \
--type=merge
    • Run one of the following commands to disable horizontal autoscaling for the service:
      • If your installation uses the CR name datarefinery-cr (for example, if this is a new installation of Data Refinery on Cloud Pak for Data version 5.0.x):
        oc patch  DataRefinery datarefinery-cr  \
--namespace ${PROJECT_CPD_INST_OPERANDS} \
--patch '{"spec":{"autoScaleConfig":false}}' \
--type=merge
      • If your installation uses the CR name datarefinery-sample (for example, if you upgraded to Data Refinery on Cloud Pak for Data version 5.0.x from an earlier version):
        oc patch  DataRefinery datarefinery-sample  \
--namespace ${PROJECT_CPD_INST_OPERANDS} \
--patch '{"spec":{"autoScaleConfig":false}}' \
--type=merge
    DataStage

    • Run the following command to enable horizontal autoscaling for the service:

      cpd-cli manage update-cr \
--component=datastage_ent \
--cpd_instance_ns=${PROJECT_CPD_INST_OPERANDS} \
--patch='{\"autoScaleConfig\":true}'

    • Run the following command to disable horizontal autoscaling for the service:

      cpd-cli manage update-cr \
--component=datastage_ent \
--cpd_instance_ns=${PROJECT_CPD_INST_OPERANDS} \
--patch='{\"autoScaleConfig\":false}'
    Data Virtualization

    You can enable autoscaling for Data Virtualization. However, it does not use the autoScaleConfig setting. For more information about scaling Data Virtualization, see Scaling Data Virtualization.

    Db2 Data Management Console

    • Run the following command to enable horizontal autoscaling for a Db2 Data Management Console service instance:

      oc patch Dmcaddon <dmcs_cr_name> \
 --namespace ${PROJECT_CPD_INST_OPERANDS} \
 --patch '{"spec":{"autoScaleConfig":true}}' \
 --type=merge

      Replace <dmcs_cr_name> with the name of the Db2 Data Management Console custom resource.

    • Run the following command to disable horizontal autoscaling for a Db2 Data Management Console service instance:

      oc patch Dmcaddon <dmcs_cr_name> \
 --namespace ${PROJECT_CPD_INST_OPERANDS} \
 --patch '{"spec":{"autoScaleConfig":false}}' \
 --type=merge

      Replace <dmcs_cr_name> with the name of the Db2 Data Management Console custom resource.

    Decision Optimization

    • Run the following command to enable horizontal autoscaling for the service:

      cpd-cli manage update-cr \
--component=dods \
--cpd_instance_ns=${PROJECT_CPD_INST_OPERANDS} \
--patch='{\"autoScaleConfig\":true}'

    • Run the following command to disable horizontal autoscaling for the service:

      cpd-cli manage update-cr \
--component=dods \
--cpd_instance_ns=${PROJECT_CPD_INST_OPERANDS} \
--patch='{\"autoScaleConfig\":false}'
    IBM Match 360 with Watson

    • Run the following command to enable horizontal autoscaling for the service:

      cpd-cli manage update-cr \
--component=match360 \
--cpd_instance_ns=${PROJECT_CPD_INST_OPERANDS} \
--patch='{\"autoScaleConfig\":true}'

    • Run the following command to disable horizontal autoscaling for the service:

      cpd-cli manage update-cr \
--component=match360 \
--cpd_instance_ns=${PROJECT_CPD_INST_OPERANDS} \
--patch='{\"autoScaleConfig\":false}'
    OpenPages

    • Run the following command to enable horizontal autoscaling for an OpenPages service instance:

      cpd-cli manage update-cr \
--component=openpages_instance \
--cpd_instance_ns=${PROJECT_CPD_INST_OPERANDS} \
--patch='{\"autoScaleConfig\":true}' \
--extra-vars=cr_name=<instance_name>

    • Run the following command to disable horizontal autoscaling for an OpenPages service instance:

      cpd-cli manage update-cr \
--component=openpages_instance \
--cpd_instance_ns=${PROJECT_CPD_INST_OPERANDS} \
--patch='{\"autoScaleConfig\":false}' \
--extra-vars=cr_name=<instance_name>
    Note: If OpenPages is installed in a tethered project, add the following parameter to these commands.
    ----tethered_instance_ns=${PROJECT_CPD_INSTANCE_TETHERED}
    Orchestration Pipelines

    • Run the following command to enable horizontal autoscaling for the service:

      cpd-cli manage update-cr \
--component=ws_pipelines \
--cpd_instance_ns=${PROJECT_CPD_INST_OPERANDS} \
--patch='{\"autoScaleConfig\":\"true\"}'

    • Run the following command to disable horizontal autoscaling for the service:

      cpd-cli manage update-cr \
--component=ws_pipelines \
--cpd_instance_ns=${PROJECT_CPD_INST_OPERANDS} \
--patch='{\"autoScaleConfig\":\"false\"}'
    Planning Analytics

    • Run the following command to enable horizontal autoscaling for a Planning Analytics service instance:

      oc patch PAServiceInstance <instance_name> \
 --namespace ${PROJECT_CPD_INST_OPERANDS} \
 --patch '{"spec":{"autoScaleConfig":true}}' \
 --type=merge

    • Run the following command to disable horizontal autoscaling for a Planning Analytics service instance:

      oc patch PAServiceInstance <instance_name> \
 --namespace ${PROJECT_CPD_INST_OPERANDS} \
 --patch '{"spec":{"autoScaleConfig":false}}' \
 --type=merge
    Product Master

    • Run the following command to enable horizontal autoscaling for the service:

      cpd-cli manage update-cr \
--component=productmaster_instance \
--cpd_instance_ns=${PROJECT_CPD_INST_OPERANDS} \
--patch='{\"autoScaleConfig\":true}'

    • Run the following command to disable horizontal autoscaling for the service:

      cpd-cli manage update-cr \
--component=productmaster_instance \
--cpd_instance_ns=${PROJECT_CPD_INST_OPERANDS} \
--patch='{\"autoScaleConfig\":false}'
    SPSS Modeler
    Tip: If you scale SPSS Modeler, you must also scale the Canvas component.

    • Run the following command to enable horizontal autoscaling for the service:

      oc patch spss spss-sample \
--namespace cpd-instance \
--patch '{"spec":{"autoScaleConfig":true}}' \
--type=merge

    • Run the following command to disable horizontal autoscaling for the service:

      oc patch spss spss-sample \
--namespace cpd-instance \
--patch '{"spec":{"autoScaleConfig":false}}' \
--type=merge
    Watson Machine Learning

    • Run the following command to enable horizontal autoscaling for the service:

      cpd-cli manage update-cr \
--component=wml \
--cpd_instance_ns=${PROJECT_CPD_INST_OPERANDS} \
--patch='{\"autoScaleConfig\":\"true\"}'

    • Run the following command to disable horizontal autoscaling for the service:

      cpd-cli manage update-cr \
--component=wml \
--cpd_instance_ns=${PROJECT_CPD_INST_OPERANDS} \
--patch='{\"autoScaleConfig\":\"false\"}'
    Watson Machine Learning Accelerator
    • Run one of the following commands to enable horizontal autoscaling for the service:

      This command uses the default custom resource name: wmla. If you use a different name, edit the command to specify the appropriate value for your environment.

      • oc project ${PROJECT_CPD_INST_OPERANDS}
export INSTANCE=`oc get wmla -n ${PROJECT_CPD_INST_OPERANDS} |grep -v NAME| awk '{print $1}'`
oc patch wmla ${INSTANCE} \
--namespace ${PROJECT_CPD_INST_OPERANDS} \
--patch='[{"op": "replace", "path": "/spec/autoScaleConfig", "value":true}]' \
--type='json'
      • cpd-cli manage update-cr \
--component=wml_accelerator \
--cpd_instance_ns=${PROJECT_CPD_INST_OPERANDS} \
--patch='{\"autoScaleConfig\":true}'
    • Run one of the following commands to disable horizontal autoscaling for the service.

      This command uses the default custom resource name: wmla. If you use a different name, edit the command to specify the appropriate value for your environment. 

      • oc project ${PROJECT_CPD_INST_OPERANDS}
export INSTANCE=`oc get wmla -n ${PROJECT_CPD_INST_OPERANDS} |grep -v NAME| awk '{print $1}'`
oc patch wmla ${INSTANCE} \
--namespace ${PROJECT_CPD_INST_OPERANDS} \
--patch='[{"op": "replace", "path": "/spec/autoScaleConfig", "value":false}]' \
--type='json'
      • cpd-cli manage update-cr \
--component=wml_accelerator \
--cpd_instance_ns=${PROJECT_CPD_INST_OPERANDS} \
--patch='{\"autoScaleConfig\":false}'
    Run the following command to verify whether horizontal autoscaling is enabled for the service:
    oc get hpa -n ${PROJECT_CPD_INST_OPERANDS}
    Watson OpenScale

    • Run the following command to enable horizontal autoscaling for the service:

      cpd-cli manage update-cr \
--component=openscale \
--cpd_instance_ns=${PROJECT_CPD_INST_OPERANDS} \
--patch='{\"autoScaleConfig\":true}'

    • Run the following command to disable horizontal autoscaling for the service:

      cpd-cli manage update-cr \
--component=openscale \
--cpd_instance_ns=${PROJECT_CPD_INST_OPERANDS} \
--patch='{\"autoScaleConfig\":false}'
    Watson Studio

    • Run the following command to enable horizontal autoscaling for the service:

      cpd-cli manage update-cr \
--component=ws \
--cpd_instance_ns=${PROJECT_CPD_INST_OPERANDS} \
--patch='{\"autoScaleConfig\":\"true\"}'

    • Run the following command to disable horizontal autoscaling for the service:

      cpd-cli manage update-cr \
--component=ws \
--cpd_instance_ns=${PROJECT_CPD_INST_OPERANDS} \
--patch='{\"autoScaleConfig\":\"false\"}'
    watsonx.ai

    • Run the following command to enable horizontal autoscaling for the service:

      cpd-cli manage update-cr \
--component=watsonx_ai_ifm \
--cpd_instance_ns=${PROJECT_CPD_INST_OPERANDS} \
--patch='{\"autoScaleConfig\":\"true\"}'

    • Run the following command to disable horizontal autoscaling for the service:

      cpd-cli manage update-cr \
--component=watsonx_ai_ifm \
--cpd_instance_ns=${PROJECT_CPD_INST_OPERANDS} \
--patch='{\"autoScaleConfig\":\"false\"}'
    watsonx Assistant
    • Run one of the following commands to enable horizontal autoscaling for the service:

      • This command uses the default custom resource name: wa. If you use a different name, edit the command to specify the appropriate value for your environment.

        oc project ${PROJECT_CPD_INST_OPERANDS}
export INSTANCE=`oc get wa -n ${PROJECT_CPD_INST_OPERANDS} |grep -v NAME| awk '{print $1}'`
oc patch wa ${INSTANCE} \
--namespace ${PROJECT_CPD_INST_OPERANDS} \
--patch='[{"op": "replace", "path": "/spec/autoScaleConfig", "value":true}]' \
--type='json'
      • cpd-cli manage update-cr \
--component=watson_assistant \
--cpd_instance_ns=${PROJECT_CPD_INST_OPERANDS} \
--patch='{\"autoScaleConfig\":true}'
    • Run one of the following commands to disable horizontal autoscaling for the service.

      This command uses the default custom resource name: wa. If you use a different name, edit the command to specify the appropriate value for your environment.

      • oc project ${PROJECT_CPD_INST_OPERANDS}
export INSTANCE=`oc get wa -n ${PROJECT_CPD_INST_OPERANDS} |grep -v NAME| awk '{print $1}'`
oc patch wa ${INSTANCE} \
--namespace ${PROJECT_CPD_INST_OPERANDS} \
--patch='[{"op": "replace", "path": "/spec/autoScaleConfig", "value":false}]' \
--type='json'
      • cpd-cli manage update-cr \
--component=watson_assistant \
--cpd_instance_ns=${PROJECT_CPD_INST_OPERANDS} \
--patch='{\"autoScaleConfig\":false}'
    watsonx Code Assistant for Red Hat Ansible Lightspeed

    • Run the following command to enable horizontal autoscaling for the service:

      cpd-cli manage update-cr \
--component=wca_ansible \
--cpd_instance_ns=${PROJECT_CPD_INST_OPERANDS} \
--patch='{\"autoScaleConfig\":true}'

    • Run the following command to disable horizontal autoscaling for the service:

      cpd-cli manage update-cr \
--component=wca_ansible \
--cpd_instance_ns=${PROJECT_CPD_INST_OPERANDS} \
--patch='{\"autoScaleConfig\":false}'
    watsonx Code Assistant for Z

    • Run the following command to enable horizontal autoscaling for the service:

      cpd-cli manage update-cr \
--component=wca_z \
--cpd_instance_ns=${PROJECT_CPD_INST_OPERANDS} \
--patch='{\"autoScaleConfig\":true}'

    • Run the following command to disable horizontal autoscaling for the service:

      cpd-cli manage update-cr \
--component=wca_z \
--cpd_instance_ns=${PROJECT_CPD_INST_OPERANDS} \
--patch='{\"autoScaleConfig\":false}'
    watsonx.data

      1. Run the following command to enable horizontal autoscaling in the addon components UI, API, and Nodeclient:

        oc patch wxdAddon wxdaddon \
--namespace ${PROJECT_CPD_INST_OPERANDS} \
--type=merge \
--patch '{"spec": {"autoScaleConfig": true}}'
      2. Complete the following steps to enable horizontal pod autoscaling of Presto workers:
        1. Run the following command to get the Presto engine name:
          oc get wxdengine

          The following example shows a returned engine name:

          NAME                  AGE
lakehouse-presto-01   75m
        2. Run the following command to enable horizontal pod autoscaling in the pods. Horizontal pod autoscaling only works with multinode deployments because the command scales worker nodes.
          oc patch wxdengine/lakehouse-presto-01 \
  --type=merge \
   -n ${PROJECT_CPD_INST_OPERANDS} \
   -p '{ "spec": { "autoScaleConfig": "true", "lakehouseDeploymentType": "multinode" } }'

      1. Run the following command to disable horizontal autoscaling in the addon components UI, API, and Nodeclient:

        oc patch wxdAddon wxdaddon \
--namespace ${PROJECT_CPD_INST_OPERANDS} \
--type=merge \
--patch '{"spec": {"autoScaleConfig": false}}'
      2. Run the following command to disable horizontal pod autoscaling in the pods:
        oc patch wxdengine/lakehouse-presto-01 \
--type=merge \
 -n ${PROJECT_CPD_INST_OPERANDS} \
 -p '{ "spec": { "autoScaleConfig": false } }'
    watsonx.governance

    5.0.1 or later

    • Run the following command to enable horizontal autoscaling for the service:

      cpd-cli manage update-cr \
--component=watsonx_governance \
--cpd_instance_ns=${PROJECT_CPD_INST_OPERANDS} \
--patch='{\"autoScaleConfig\":true}'

    • Run the following command to disable horizontal autoscaling for the service:

      cpd-cli manage update-cr \
--component=watsonx_governance \
--cpd_instance_ns=${PROJECT_CPD_INST_OPERANDS} \
--patch='{\"autoScaleConfig\":false}'