Important:

IBM Cloud Pak® for Data Version 4.7 will reach end of support (EOS) on 31 July, 2025. For more information, see the Discontinuance of service announcement for IBM Cloud Pak for Data Version 4.X.

Upgrade to IBM Software Hub Version 5.1 before IBM Cloud Pak for Data Version 4.7 reaches end of support. For more information, see Upgrading IBM Software Hub in the IBM Software Hub Version 5.1 documentation.

Scaling a deployment

When you create an online deployment for a model, function, or Shiny app from a deployment space or programmatically, a single copy of the asset is deployed by default. To increase scalability and availability, you can increase the number of copies (replicas) by editing the configuration of the deployment. Additional replicas allow for a larger volume of scoring requests.

Deployments can be scaled in the following ways:

  • Update the configuration for a deployment in a deployment space.
  • Programmatically, using the Watson Machine Learning Python client library, or the Watson Machine Learning REST APIs.
Important:

You can scale up to ten copies by using the UI. If a larger number of copies are required then use the API to scale your deployment.

Increasing the number of replicas of an online deployment from a space

  1. Click to open the Deployment tab of your deployment space.

  2. Click Edit configuration on the action menu for the deployment name.

  3. Change the number of replicas and save your change.

    Scale a deployment

    Tip: You can also update the number of replicas from the information sheet for the deployment.

    Follow these steps to update the number of replicas:

  4. Click the deployment name to view the details.

  5. Click the Edit icon in the information sheet to edit the number of copies.
    Updating a deployment configuration

Increasing the number of replicas of a deployment programmatically

To view or run a working sample of scaling a deployment programmatically, you can increase the number of replicas in the metadata for a deployment.

Python example

This example uses the Python client to set the number of replicas to 3.

change_meta = {
                client.deployments.ConfigurationMetaNames.HARDWARE_SPEC: {
                                       "name":"S",
                                       "num_nodes":3}
            }

client.deployments.update(<deployment_id>, change_meta)

The HARDWARE_SPEC value includes a name because the API requires a name or an id to be provided. However, this argument is disregarded for online deployments with the following frameworks: Spark, PMML, Scikit-learn (non-custom image), XGBoost (non-custom image), PyTorch (non-custom image).

REST API example

curl -k -X PATCH -d '[ { "op": "replace", "path": "/hardware_spec", "value": {  "name": "S", "num_nodes": 2  } } ]' <Deployment end-point URL>

You must specify a name for the hardware_spec value, but the argument is not applied for scaling.

Parent topic: Managing predictive deployments