Creating and uploading a custom image

Perform these steps to create and upload a custom image for Python models and functions in Watson Machine Learning.

Required role: You must be a Cloud Pak for Data cluster administrator.

Note: The samples provided here use Podman, but you can use any other Docker-compatible image builder.

Creating and uploading a custom Python image

Required role: You must have root privileges on your machine to execute the commands that build a custom image.

After you have downloaded the image for the runtime that you want to customize, you can build a new custom image by adding your customizations to the downloaded image. You start with the base image and install the libraries and packages that you need. You do this by adding your customizations to a Dockerfile.

1. Prepare the Dockerfile

   Notes:

   • Do not change the contents in the USER directives as they impact how the container will later run in the cluster
   • You must be root to install OS packages and to modify the kernel specifications
   • Use the microdnf tool to install the OS libraries.

2. Create a custom Watson Machine Learning image.

   Sample Dockerfile for CPD 4.0.x:

   ARG  base_image_tag
   FROM ${base_image_tag}
   # For installing OS packages, use root:root
   # e.g.
   USER root:root
   RUN yum install -y gdb
   # For installing to conda site-packages,
   # use wmlfuser:condausers
   # e.g.
   USER wmlfuser:condausers
   RUN umask 002 && \
   pip install gensim
   #
   # Alternatively, for Installing to location other than conda site-packages.
   # It can be installed to
   #    --target <any_file_system_dir_path>
   # Provided <any_file_system_dir_path> full path is accessible to any user with group "condausers"
   #
   # Important:
   #     The <any_file_system_path> must then be added to the environment variable PYTHONPATH
   #
   # e.g 
   # use wmlfuser:condausers to install to the path /home/wmlfuser/mypkgs and add
   # the path to PYTHONPATH environment variable
   #
   USER wmlfuser:condausers
   RUN umask 002 && \
   mkdir -p /home/wmlfuser/mypkgs && \
   pip install pydot --target /home/wmlfuser/mypkgs
   ENV PYTHONPATH=$PYTHONPATH:/home/wmlfuser/mypkgs
   # If pip --user is used for installing package.
   # The installation path needs to be explicitly added to PYTHONPATH
   # e.g.
   #    
   #    for user wmlfuser:condausers it by default goes to user install directory
   #    (See the Python documentation for site.USER_BASE for full details.)
   #    use `python -m site --user-site` command to get the path.
   #
   #    for wml-deployment-runtime-py39-1, it goes to
   #     /home/wmlfuser/.local/lib/python3.9/site-packages
   #
   #
   

   where <source-registry-url> is the <external-image-registry-url> and <source-namespace> is the <namespace-in-external-registry> described in the section Downloading the base image.

   For Podman:

   podman build -t  <custom-image-name>:<custom-image-tag> \
   --build-arg base_image_tag=<source-registry-url>/<source-namespace>/<base-image-name>:<base-image-version> \
   -f <path_to_dockerfile>
   

   Note: If you are updating an exsiting CPD 3.5 image, refer to the sample dockerfile in Updating a custom image.

3. Upload the custom image

   The custom image must be uploaded to the OpenShift Cluster internal image registry or the private container registry which was used at the time of installation.

Adding customizations for SPSS models

To add customizations for SPSS, run the Docker file for installing the Exasol driver in SPSS:

Note: For <SPSS_RUNTIME_IMAGE>, use the image name from Downloading a custom image configuration file for SPSS.

FROM <SPSS_RUNTIME_IMAGE> AS custom_image 
 # Install Exasol ODBC driver 
 RUN mkdir -p /opt/exasol
 RUN wget -O /opt/exasol/EXASOL_ODBC.tar.gz \
           https://www.exasol.com/support/secure/attachment/175398/EXASOL_ODBC-${EXASOL_ODBC_VERSION}.tar.gz
              RUN ls -l /opt/exasol/EXASOL_ODBC.tar.gz
RUN cd /opt/exasol \
 && tar -xzf EXASOL_ODBC.tar.gz
RUN mv  /opt/exasol/EXASolution_ODBC-${EXASOL_ODBC_VERSION}/* /opt/exasol
RUN rm -rf /opt/exasol/EXASolution_ODBC-${EXASOL_ODBC_VERSION} \
    && rm -rf /opt/exasol/EXASOL_ODBC.tar.gz
# enable Exasol ODBC Driver
RUN sed -i '2iExasol Client=Installed' ${SDAPDRIVER_HOME}/odbcinst.ini
USER ${BASE_USER_NAME}

Uploading the custom image to the internal image registry

Required role: You must be a Cloud Pak for Data cluster administrator.

Notes:

• The internal image registry is image-registry.openshift-image-registry.svc:5000
• To access the internal registry, you must access it through the default route registry.
• Make sure that the defaultRoute is enabled. You can check if the this query returns true oc get configs.imageregistry.operator.openshift.io/cluster -o=jsonpath='{.spec.defaultRoute}'
• If the defaultRoute is not enabled, enable it before proceeding. You can get more information on how to enable the default route in Openshift documentation.

1. Get the default route:

    DEFAULT_ROUTE=$(oc get route default-route -n openshift-image-registry -o=jsonpath='{ .spec.host }')
    echo $DEFAULT_ROUTE
   

1. Log in to the image registry:

    podman login -u kubeadmin -p $(oc whoami -t) --tls-verify=false $DEFAULT_ROUTE
   

2. Get the Watson Machine Learning namespace:

    WML_NAMESPACE=$(oc get wmlbase wml-cr -o=jsonpath='{.metadata.namespace}')
    echo $WML_NAMESPACE
   

3. Tag and push the custom image to the default route:

    CUSTOM_IMAGE=<custom-image-name>:<custom-image-tag> 
    podman tag $CUSTOM_IMAGE $DEFAULT_ROUTE/$WML_NAMESPACE/$CUSTOM_IMAGE
    podman push $DEFAULT_ROUTE/$WML_NAMESPACE/$CUSTOM_IMAGE  --tls-verify=false
   

   Even though pushed to default route, the image will go to the internal image registry. The target custom image name with URL that you must use in registration is

    CUSTOM_IMAGE_WITH_URL=image-registry.openshift-image-registry.svc:5000/$WML_NAMESPACE/$CUSTOM_IMAGE
    echo $CUSTOM_IMAGE_WITH_URL
   

Uploading the custom image to the private container registry

Required role: You must be a Cloud Pak for Data cluster administrator.

Notes:

• If the private container registry that was used at the time of Watson Machine Learning installation has write access, then the custom image can be uploaded to the private container registry.
• A CPD cluster pulls images directly from the IBM® Entitled Registry or from a private container registry. You can read more about this in Configuring your cluster to pull images.
• If the Watson Machine Learning installation uses IBM® Entitled Registry, then the custom image must be uploaded to the internal docker registry because the IBM® Entitled Registry is a read only registry.

Log in and then tag and push the custom image to the private container registry:

CUSTOM_IMAGE=<custom-image-name>:<custom-image-tag> 
podman login -u <username> -p <>-p <password> <external-image-registry-url>
podman tag $CUSTOM_IMAGE \
<external-image-registry-url>/<namespace-in-external-registry>/$CUSTOM_IMAGE
podman push <external-image-registry-url>/<namespace-in-external-registry>/$CUSTOM_IMAGE

The target custom image name with URL that needs to be used in registration of the custom image is:

CUSTOM_IMAGE_WITH_URL=<external-image-registry-url>/<namespace-in-external-registry>/$CUSTOM_IMAGE
echo $CUSTOM_IMAGE_WITH_URL

Next steps

Creating a new base software specification for the custom image

Parent topic: Working with custom images