Migrating Python 3.7 and Python 3.8 environments from Cloud Pak for Data 4.0

When you upgrade to Cloud Pak for Data 4.6, only the following Python runtimes are available:

  • IBM Runtime 22.2 on Python 3.10 (with and without GPU)
  • IBM Runtime 22.1 on Python 3.9 (with and without GPU)
  • JupyterLab with IBM Runtime 22.2 on Python 3.10
  • JupyterLab with IBM Runtime 22.1 on Python 3.9
  • Default Spark 3.3 & Python 3.10
  • Default Spark 3.2 & Python 3.9

You can select any of the default environments with Python 3.10 or 3.9 that are included in Watson Studio, or create custom environments that use Python 3.10 or 3.9. You can also continue using custom environments that use custom runtime images with Python 3.7 or 3.8, after you have performed a migration step that is detailed in the following section.

However, you can no longer select any default Python 3.7 or 3.8 environments as these are no longer included in Watson Studio, or start notebooks or JupyterLab, run jobs, or create custom runtime images using these Python versions. Existing custom environments in which you selected Default Python 3.7 or Default Python 3.8 as software version will also not work after you have upgraded.

What does this mean for existing Python notebooks and jobs?

After you have upgraded to Cloud Pak for Data 4.6:

  • The Default Python 3.7 and Default Python 3.8 environments are no longer available. Notebooks and jobs that use these environments will not run.
  • Custom environments with Default Python 3.7 or Default Python 3.8 as software version will be migrated, but all notebooks and jobs associated with these environments will not run. Also, these custom environments will become corrupted when they are exported from a project and can't be used if imported to a new project. Issues can also occur in Git-based projects.
  • Custom environments that you created based on custom runtime images will still be valid. Notebooks and jobs associated with these environments will still run.

Before you upgrade to Cloud Pak for Data 4.6, you must consider the following:

  1. Evaluate the library differences between the environment runtime you are currently using and those in the 22.x Runtime environment, and the implications for code changes that you would need to make.

  2. If you are on Cloud Pak for Data 4.0.7 and earlier, and you want to continue using a Default Python 3.7 or Default Python 3.8 runtime:

    1. Download the appropriate runtime configuration file. See Downloading the runtime configuration.
      • For Notebooks, use one of the following configuration files:
        • jupyter-py38-server.json
        • jupyter-py37-server.json
        • jupyter-py38gpu-server.json
        • jupyter-py37gpu-server.json
      • For Jupyterlab, use:
        • jupyter-lab-py38-server.json
        • jupyter-lab-py37-server.json
        • jupyter-lab-py38gpu-server.json
        • jupyter-lab-py37gpu-server.json
    2. Rename the runtime configuration, for example, to py38-cpd40-server.json.
    3. Upload the file to the Cloud Pak for Data cluster. See Step 7 in Uploading the custom configuration.
    1. Switch to using an environment with this image in your notebooks and jobs. For details, see Changing the environment of a notebook.
    2. Test run the notebooks and jobs.
    3. If they run successfully, you can upgrade to Cloud Pak for Data 4.6. Your custom environments with this change will be migrated.

  3. If you determined that your existing notebooks and job will work with a Runtime 22.x environment (for Python 3.10 or 3.9):

    1. Upgrade to Cloud Pak for Data 4.6.
    2. Go to your existing notebooks and jobs, and select a Runtime 22.x environment.

  4. After upgrading to Cloud Pak for Data 4.6, delete any remaining Jupyter notebook runtimes with Python 3.7 and 3.8:

    1. Run the following command to see which runtimes are installed:

      $ cpd-cli manage get-cr-status \
--cpd_instance_ns=${PROJECT_CPD_INSTANCE} \
--components=ws_runtimes

      For information on using the variable PROJECT_CPD_INSTANCE, see Setting up installation environment variables.

    2. If one of the following environments is included in the output:

      • ibm-cpd-ws-runtime-py37
      • ibm-cpd-ws-runtime-py37gpu
      • ibm-cpd-ws-runtime-py38
      • ibm-cpd-ws-runtime-py38gpu

      Delete the runtime by running the following command:

      oc delete -n ${PROJECT_CPD_INSTANCE} --ignore-not-found NotebookRuntime ibm-cpd-ws-runtime-py37 ibm-cpd-ws-runtime-py37gpu ibm-cpd-ws-runtime-py38 ibm-cpd-ws-runtime-py38gpu

Parent topic: Environments