Working with Watson Machine Learning Accelerator notebooks in IBM Cloud Pak for Data

Use IBM Watson® Machine Learning Accelerator notebooks to run elastic distributed training workloads in IBM® Cloud Pak for Data.

  1. Before using Watson Machine Learning Accelerator notebooks, complete the following task:
  2. Submitting workloads using Watson Machine Learning Accelerator notebooks, see Using elastic distributed training API in notebooks.

Considerations for upgrading from IBM Cloud Pak for Data Version 4.8 to Version 5.0.x

If you are upgrading from IBM Cloud Pak for Data Version 4.8 to Version 5.0.x, you will need to update your notebooks. After upgrading, you must:
  • Edit the notebook runtime

Edit the notebook runtime

From IBM Cloud Pak for Data, follow these steps to find and edit a notebook runtime:
  1. In the project navigation trail, click your project name to return to your project.

  2. Click the Assets tab to find the notebook.

  3. When you click the notebook, it will open in READ ONLY mode.

  4. To edit the notebook, click the pencil icon .

  5. Click the Information icon to open the Information panel.

  6. On the General tab, edit the name and description of the notebook.

  7. Click the Environment tab to change the environment used to run the notebook and update the runtime to either Runtime 24.1 on Python 3.11 (recommended) or GPU Runtime 24.1 on Python 3.11.

Considerations for upgrading from IBM Cloud Pak for Data Version 4.7 to Version 5.0.x

If you are upgrading from IBM Cloud Pak for Data Version 4.7 to Version 5.0.x, you will need to update your notebooks. After upgrading, you must:
  • Edit the notebook runtime
  • Edit the runtime code

Edit the notebook runtime

From IBM Cloud Pak for Data, follow these steps to find and edit a notebook runtime:
  1. In the project navigation trail, click your project name to return to your project.

  2. Click the Assets tab to find the notebook.

  3. When you click the notebook, it will open in READ ONLY mode.

  4. To edit the notebook, click the pencil icon .

  5. Click the Information icon to open the Information panel.

  6. On the General tab, edit the name and description of the notebook.

  7. Click the Environment tab to change the environment used to run the notebook and update the runtime to either Runtime 24.1 on Python 3.11 (recommended) or GPU Runtime 24.1 on Python 3.11.

Edit notebook code

To ensure that your notebook code runs using notebook runtime Runtime 24.1 on Python 3.11 or GPU Runtime 24.1 on Python 3.11, you must:
  • Update import FabricModel statement
  • Remove references to DATA_DIR

Update import FabricModel statement

The following notebook code must be updated.

Change the following import statement from:
from fabric_model import FabricModel
To:
from pyedt.fabric_model import FabricModel

Update references to DATA_DIR

Update any instances of DATA_DIR in the notebook code as it is no longer supported. Data must now be downloaded from a directory.

For example, change the following code from:
"DATA_DIR = os.getenv("DATA_DIR")"
"print("DATA_DIR: " + os.getenv("DATA_DIR"))"
To:
"DATA_DIR = os.getenv("DATA_DIR", "/gpfs/mydatafs")"
"print("DATA_DIR: " + DATA_DIR)"
Example: to download data from a directory, replace the following by removing the following code:
datasets.MNIST(DATA_DIR, download=True)

def getDatasets():
    return (datasets.MNIST(DATA_DIR, train=True, download=False, transform=transforms.Compose([
                           transforms.ToTensor()
                       ])),
            datasets.MNIST(dataset_dir, train=False, download=False, transform=transforms.Compose([
                           transforms.ToTensor()
                       ]))
            )
Replacing the code referencing DATA_DIR with the following code:
dataset_dir = <your_storage_volume_path>
def getDatasets():
    return (datasets.MNIST(dataset_dir, train=True, download=True, transform=transforms.Compose([
                           transforms.ToTensor()
                       ])),
            datasets.MNIST(dataset_dir, train=False, download=Ture, transform=transforms.Compose([
                           transforms.ToTensor()
                       ]))
            )

For additional examples, see IBM Git.

Create a notebook

To create a notebook in Cloud Pak for Data, see Creating notebooks.

Open a notebook

You can find and open notebooks from the Assets page of the project.

To open the notebook, click on the notebook for it to open in edit mode.