Working with Watson Machine Learning Accelerator notebooks in IBM Cloud Pak for Data
Use IBM Watson® Machine Learning Accelerator notebooks to run elastic distributed training workloads in IBM® Cloud Pak for Data.
- Before using Watson Machine Learning Accelerator notebooks, complete the following task:
- Submitting workloads using Watson Machine Learning Accelerator notebooks, see Using elastic distributed training API in notebooks.
Considerations for upgrading from IBM Cloud Pak for Data Version 4.8 to Version 5.0.x
- Edit the notebook runtime
Edit the notebook runtime
-
In the project navigation trail, click your project name to return to your project.
-
Click the Assets tab to find the notebook.
-
When you click the notebook, it will open in
READ ONLYmode. -
To edit the notebook, click the pencil icon .
-
Click the Information icon to open the Information panel.
-
On the General tab, edit the name and description of the notebook.
-
Click the Environment tab to change the environment used to run the notebook and update the runtime to either Runtime 24.1 on Python 3.11 (recommended) or GPU Runtime 24.1 on Python 3.11.
Considerations for upgrading from IBM Cloud Pak for Data Version 4.7 to Version 5.0.x
- Edit the notebook runtime
- Edit the runtime code
Edit the notebook runtime
-
In the project navigation trail, click your project name to return to your project.
-
Click the Assets tab to find the notebook.
-
When you click the notebook, it will open in
READ ONLYmode. -
To edit the notebook, click the pencil icon .
-
Click the Information icon to open the Information panel.
-
On the General tab, edit the name and description of the notebook.
-
Click the Environment tab to change the environment used to run the notebook and update the runtime to either Runtime 24.1 on Python 3.11 (recommended) or GPU Runtime 24.1 on Python 3.11.
Edit notebook code
- Update import FabricModel statement
- Remove references to DATA_DIR
Update import FabricModel statement
The following notebook code must be updated.
from fabric_model import FabricModelTo:from pyedt.fabric_model import FabricModelUpdate references to DATA_DIR
Update any instances of DATA_DIR in the notebook code as it is no longer supported. Data must now be downloaded from a directory.
"DATA_DIR = os.getenv("DATA_DIR")"
"print("DATA_DIR: " + os.getenv("DATA_DIR"))"To:"DATA_DIR = os.getenv("DATA_DIR", "/gpfs/mydatafs")"
"print("DATA_DIR: " + DATA_DIR)"datasets.MNIST(DATA_DIR, download=True)
def getDatasets():
return (datasets.MNIST(DATA_DIR, train=True, download=False, transform=transforms.Compose([
transforms.ToTensor()
])),
datasets.MNIST(dataset_dir, train=False, download=False, transform=transforms.Compose([
transforms.ToTensor()
]))
)Replacing the code referencing DATA_DIR with the following code:
dataset_dir = <your_storage_volume_path>
def getDatasets():
return (datasets.MNIST(dataset_dir, train=True, download=True, transform=transforms.Compose([
transforms.ToTensor()
])),
datasets.MNIST(dataset_dir, train=False, download=Ture, transform=transforms.Compose([
transforms.ToTensor()
]))
)
For additional examples, see IBM Git.
Create a notebook
To create a notebook in Cloud Pak for Data, see Creating notebooks.
Open a notebook
You can find and open notebooks from the Assets page of the project.
To open the notebook, click on the notebook for it to open in edit mode.