Building deployment runtime images

Create a custom Docker image with libraries and packages that are required for your deployments, then use the custom image to deploy your Watson Machine Learning assets.

You can build custom images based on deployment runtime images available in IBM Watson Machine Learning. The images contain preselected open source libraries and selected IBM libraries. By building custom images, you can optimize the standard software configuration of a runtime for your application needs. You can also use custom images in air-gapped environments with requirements that forbid you to show any operations to the internet.

Notes for working with custom images

  • The way that you create a custom image depends on whether you are building an image for Python-based models and functions or an SPSS image. For more information, see Creating a custom image.
  • These custom image types are not compatible with 4.5.0 or later releases of Cloud Pak for Data:
    • Images that were created in Cloud Pak for Data version 3.5
    • Images that were created before the 4.0.6 release of Cloud Pak for Data 4.x

If your custom image belongs to one of these categories, you must rebuild your image. Use the new base images published in the current release.

Rebuilding an existing image

  1. Download the base image.
  2. Rebuild the image with required libraries.
  3. Upload the rebuilt image to the local Cloud Pak for Data registry.
  4. Update the existing runtime-definition file with the new image.
  5. Perform online or batch scoring on the existing deployment.
Note:

If your deployed model uses a custom image that was created in one of previous releases of Cloud Pak for Data, you don't need to create a fresh deployment.

Training the model

If you want to train your SPSS model that uses a custom image, you can start with Building custom images to install ODBC drivers.

If you want to train your Python models or functions that use a custom image, you can start with [Creating and training with custom image for Watson Studio](IBM Software Hub/build-cust-images.html){: external}. The Watson Studio image cannot be used for deployment in Watson Machine Learning, though. For deployment, create a custom image for Watson Machine Learning, as described in Creating a Watson Machine Learning custom image.

Supported frameworks for custom images

This table lists frameworks that are supported by the Watson Machine Learning runtime image.

Framework Online Batch
Scikit-learn Yes Yes
XGBoost Yes Yes
TensorFlow Yes Yes
PyTorch Yes Yes
Python function Yes Programmatic only
Inline payload only
Python scripts No Yes

For versions that are supported for each framework, see Supported machine learning tools, libraries, frameworks, and software specifications.

This table lists the frameworks that are supported by the SPSS runtime image:

Framework Online Batch
SPSS No Yes

Creating and registering a custom image

Required roles: You must be an admin of the Cloud Pak for Data instance to download a runtime definition and register a runtime definition for a new image. You must have root privileges on your machine to execute the commands that build a custom image.

Important: You must carry out the following steps on all existing custom images each time you upgrade Cloud Pak for Data to rebase the image with the latest Cloud Pak for Data runtime image, which includes all of the latest fixes, including security fixes.

To create and register a custom image:

  1. Get the registry URL, download the configuration file for your runtime image, and download the image in the configuration:

    1. Getting the registry URL to use for Docker commands and in scripts. The Watson Machine Learning deployment images are stored in a Docker image registry. In Watson Machine Learning, you can only use an external registry outside of the Cloud Pak for Data OpenShift cluster. For more information, see Getting registry URL.

      To use that registry, you need the URL to the external registry that was used during the installation of Cloud Pak for Data. You use the same URL for all commands and in all scripts that you run.

    2. Downloading the configuration file for the runtime image that you want to customize. For more information, see Downloading the configuration file.

    3. Downloading the image in the definition. For more information, see Downloading the image.

  2. Add customizations and build a new image. For more information, see Creating a custom image.
  3. Push the image to the container server. For more information, see Pushing the image to the registry.
  4. Create a base software specification for the custom image. For more information, see Creating a base software specification for the custom image.
  5. Change and upload the configuration file. For more information, see Uploading the custom configuration.

Using the image for deployments

To use the custom image, follow these steps:

  1. Save the trained model that you want to deploy with the custom software specification that is defined for the custom image to the Watson Machine Learning repository.
  2. Deploy the model

Sample notebook

For sample notebooks that demonstrate how to use custom images to deploy a Python function, see Watson Machine Learning Python client samples and examples.

More considerations

If you use custom images, you must consider the following aspects:

  • Custom images for deployment can be built for Python with CPU environments only.

  • Make sure all updates that are made to the available runtime images in Watson Machine Learning, including all security updates, are also made to your custom images. Watch out for new fix packs or any related information. When new versions of Cloud Pak for Data are released, consider rebuilding all of your custom images.

  • Watson Machine Learning custom images can’t be used in Watson Studio or Data Refinery. They can be used in Watson Machine Learning only.

  • Watson Machine Learning uses uwsgi to run the deployment server. Make sure any extra packages and libraries that you install do not break uwsgi.

  • To get the list of Watson Machine Learning custom images, enter:

    oc get pods -l type=wml

  • After you install any package for Watson Machine Learning, make it available to wmlfuser:condausers.

  • User Python code or packages that are copied to the container have read/write permissions for all users.

  • Pod spawning is taken care of by the deployment manager:

    oc get pods | grep wml-deployment-manager

    Therefore, pod spawning errors are more often captured in the deployment manager log.

    oc logs wml-deployment-manager-0 > rm.log

Parent topic: Customizing deployment runtimes