Building deployment runtime images

Create a custom Docker image with libraries and packages that are required for your deployments, then use the custom image to deploy your Watson Machine Learning assets.

You can build custom images based on deployment runtime images available in IBM Watson Machine Learning. The images contain preselected open source libraries and selected IBM libraries. By building custom images, you can optimize the standard software configuration of a runtime for your application needs. You can also use custom images in air-gapped environments with requirements that forbid you to show any operations to the internet.

Notes for working with custom images

  • The way that you customize an image depends on whether you are:

    • Customizing an image for Python-based models and functions
    • Customizing an SPSS image
    • Working in an air-gapped environment
    • Creating a custom image for a custom foundation model

  • These custom image types are not compatible with 4.5.0 or later releases of Cloud Pak for Data:

    • Images that were created in Cloud Pak for Data version 3.5
    • Images that were created before the 4.0.6 release of Cloud Pak for Data 4.x

    If your existing custom image belongs to one of these categories, follow the steps that are listed in Rebuilding an existing image.

  • The standard workflow for customizing runtime images is listed in Creating and registering a custom image. If you want to bring back a discontinued Spark 3.3 image, you'll have to follow the same flow, but skip some of the steps.

Supported frameworks for custom images

This table lists frameworks that are supported by the Watson Machine Learning runtime image.

Frameworks that are supported by the Watson Machine Learning runtime image
Framework Online Batch
Scikit-learn Yes Yes
XGBoost Yes Yes
TensorFlow Yes Yes
PyTorch Yes Yes
Python function Yes Programmatic only
Inline payload only
Python scripts No Yes

This table lists the frameworks that are supported by the SPSS runtime image:

Frameworks that are supported by the SPSS runtime image
Framework Online Batch
SPSS No Yes

This table lists the frameworks that are supported by the Spark runtime image:

Frameworks that are supported by the Spark runtime image
Framework Online Batch
Spark Yes No

For available framework versions, see Supported machine learning tools, libraries, frameworks, and software specifications.

Rebuilding an existing image

Required roles: You must be an admin of the Cloud Pak for Data instance to download a runtime definition and register a runtime definition for the image.

  1. Download the base image.
  2. Rebuild the image by using required libraries
  3. Upload the image to the container registry.
  4. Edit and upload the runtime definition.
  5. Perform online or batch scoring on the existing deployment.
Note:

If your deployed model uses a custom image that was created in one of previous releases of Cloud Pak for Data, you don't need to create a fresh deployment.

Creating and registering a custom image

Required roles:

  • You must be an admin of the Cloud Pak for Data instance to download a runtime definition and register a runtime definition for the image.
  • You must have root privileges on your machine to execute the commands that build a custom image.
Important: You must carry out the following steps on all existing custom images each time you upgrade Cloud Pak for Data to rebase the image with the latest Cloud Pak for Data runtime image, which includes all of the latest fixes, including security fixes.
Note:

If you want to add Spark 3.3 (discontinued) to the list of available software specifications, the steps that you take depend on your situation:

  • If you work in an air-gapped environment, skip the "Add customizations and build a new image" step.
  • In other environments, follow these steps: "Create a base software specification for the custom image" and "Change and upload the configuration file".

To create and register a custom image:

  1. Get the URL of the registry where the image is located.
  2. Download the configuration file for the runtime image that you want to customize.
  3. Download the base image.
  4. Add customizations and build a new image.
  5. Upload the image to the container registry.
  6. Create a base software specification for the custom image.
  7. Edit and upload the runtime definition.

Training models by using custom images

To train a model, use Watson Studio:

  • If you want to use a custom image to train Python models or functions, see [Creating and training with custom image for Watson Studio](IBM Software Hub/svc-wsruntimes/build-cust-images.html){: external}.
  • If you want to train your SPSS model that uses a custom image, see Building custom images to install R and Python packages.
Note:

The Watson Studio images cannot be used to deply models.

Using the image for deployments

To use the custom image, follow these steps:

  1. Save the trained model that you want to deploy with the custom software specification that is defined for the custom image to the Watson Machine Learning repository.
  2. Deploy the model

Sample notebook

For sample notebooks that demonstrate how to use custom images to deploy a Python function, see Watson Machine Learning Python client samples and examples.

More considerations

If you use custom images, you must consider the following aspects:

  • Custom images for deployment can be built for Python with CPU environments only.

  • Make sure that all updates that are made to the available runtime images in Watson Machine Learning, including all security updates, are also made to your custom images. Watch out for new fix packs or any related information. When new versions of Cloud Pak for Data are released, consider rebuilding all of your custom images.

  • Watson Machine Learning custom images can’t be used in Watson Studio or Data Refinery. They can be used in Watson Machine Learning only.

  • Watson Machine Learning uses uwsgi to run the deployment server. Make sure any extra packages and libraries that you install do not break uwsgi.

  • To get the list of Watson Machine Learning custom images, type:

    oc get pods -l type=wml

  • After you install any package for Watson Machine Learning, make it available to wmlfuser:condausers.

  • User Python code or packages that are copied to the container must have read/write permissions for all users.

  • Pod spawning is taken care of by the deployment manager:

    oc get pods | grep wml-deployment-manager

    Therefore, pod spawning errors are more often captured in the deployment manager log.

    oc logs wml-deployment-manager-0 > rm.log

Parent topic: Customizing deployment runtimes