Building custom deployment images

Create a custom Docker image with libraries and packages that are required for your deployments, then use the custom image to deploy your Watson Machine Learning assets.

You can build custom images based on deployment runtime images available in Watson Machine Learning. The images contain preselected open source libraries and selected IBM libraries. By building custom images, you can optimize the standard software configuration of a runtime for your application needs. You can also use custom images in air-gapped environments with requirements that forbid you to show any operations to the Internet.

Before you begin

The way that you customize an image depends on whether you are:
  • Customizing an image for Python-based models and functions
  • Customizing an SPSS image
  • Working in an air-gapped environment
  • Creating a custom image for a custom foundation model
These custom image types are not compatible with 4.5.0 or later releases of IBM® Software Hub:
  • Images that were created in Cloud Pak for Data version 3.5
  • Images that were created before the 4.0.6 release of Cloud Pak for Data
If your existing custom image belongs to one of these categories, follow the steps that are listed in Rebuilding an existing image.

The standard workflow for customizing runtime images is listed in Creating and registering a custom image. If you want to bring back a discontinued Spark 3.3 image, you'll have to follow the same flow, but skip some of the steps.

Supported frameworks for custom images

Table 1. Frameworks that are supported by the Watson Machine Learning runtime image
Framework Online Batch
Scikit-learn Yes Yes
XGBoost Yes Yes
TensorFlow Yes Yes
PyTorch Yes Yes
Python function Yes

Programmatic only

Inline payload only
Python scripts No Yes
Table 2. Frameworks that are supported by the SPSS runtime image
Framework Online Batch
SPSS No Yes
Table 3. Frameworks that are supported by the Spark runtime image
Framework Online Batch
Spark Yes No

Rebuilding an existing image

Required role: You must be an admin of the IBM Software Hub instance to download a runtime definition and register a runtime definition for the image.

To rebuild an existing image:
  1. Download the base image
  2. Rebuild the image by using required libraries
  3. Upload the image to the container registry
  4. Edit and upload the runtime definition
  5. Perform online or batch scoring on the existing deployment
Note: If your deployed model uses a custom image that was created in one of previous releases of IBM Software Hub, you don't need to create a fresh deployment.

Creating and registering a custom image

Required roles:
  • You must be an admin of the IBM Software Hub instance to download a runtime definition and register a runtime definition for the image.
  • You must have root privileges on your machine to execute the commands that build a custom image.
Important:

You must carry out the following steps on all existing custom images each time you upgrade Cloud Pak for Data to rebase the image with the latest Cloud Pak for Data runtime image, which includes all of the latest fixes, including security fixes.

Note:
If you want to add Spark 3.3 (discontinued) to the list of available software specifications, the steps that you take depend on your situation:
  • If you work in an air-gapped environment, skip the Add customizations and build a new image step.
  • In other environments, follow these steps: Create a base software specification for the custom image and Edit and upload the configuration file.
To create and register a custom image:

Training models by using custom images

To train a model, use Watson Studio:
Note: The Watson Studio images cannot be used to deploy models.

Using the image for deployments

To use the custom image, follow these steps:
  1. Save the trained model that you want to deploy with the custom software specification that is defined for the custom image to the Watson Machine Learning repository
  2. Deploy the model

More considerations

If you use custom images, you must consider the following aspects:
  • Custom images for deployment can be built for Python with CPU environments only.
  • Make sure that all updates that are made to the available runtime images in Watson Machine Learning, including all security updates, are also made to your custom images. Watch out for new fix packs or any related information. When new versions of Cloud Pak for Data are released, consider rebuilding all of your custom images.
  • Watson Machine Learning custom images can’t be used in Watson Studio or Data Refinery. They can be used in Watson Machine Learning only.
  • Watson Machine Learning uses uwsgi to run the deployment server. Make sure any extra packages and libraries that you install do not break uwsgi.
  • To get the list of Watson Machine Learning custom images, type this command: 
    oc get pods -l type=wml
  • After you install any package for Watson Machine Learning, make it available to wmlfuser:condausers.
  • User Python code or packages that are copied to the container must have read/write permissions for all users.
  • Pod spawning is taken care of by the deployment manager:
    oc get pods | grep wml-deployment-manager
    Therefore, pod spawning errors are more often captured in the deployment manager log:
    oc logs wml-deployment-manager-0 > rm.log