Table of contents

Saving an AutoAI generated notebook (Watson Machine Learning)

To view the code that created a particular experiment, or interact with the experiment programmatically, you can save an experiment as a notebook. If you build an AutoAI experiment that is created from a single data source (not joined data) and is not a time series experiment, you can also save an individual pipeline as a model.

Note: Notebooks created for experiments that use joined data are available as beta and should not be used in a production environment.

Working with AutoAI-generated notebooks

When you save an experiment or a pipeline as notebook, you can:

  • Access the saved notebooks from the Notebooks section on the Assets page of your project.
  • Review the code to understand the transformations applied to build the model. This increases confidence in the process and contributes to explainable AI practices.
  • Enter your own authentication credentials using the template provided.
  • Use and run the code within Watson Studio, or download the notebook code to use in another notebook server. No matter where you use the notebook, it automatically installs all required dependencies, including libraries for:
    • xgboost
    • lightgbm
    • scikit-learn
    • autoai-libs
    • ibm-watson-machine-learning

Notes:

  • Auto-generated notebook code will excute succcessfully as written. Modifying the code or changing the input data can adversely affect the code. If you want to make a significant change, consider retraining the experiment using AutoAI.
  • For more information on the estimators, or algorithms, and transformers that are applied to your data to train an experiment and create pipelines, refer to Implementation details.

Saving an experiment as a notebook

Save all of the code for an experiment to view the transformations and optimizations applied to create the model pipelines.

What is included with the experiment notebook

The experiment notebook provides annotated code so you can:

  • Interact with trained model pipelines
  • Access model details programatically (including feature importance and machine learning metrics).
  • Visualize each pipeline as a graph, with each node documented, to provide transparency
  • Compare pipelines
  • Download selected pipelines and test locally
  • Create a deployment and score the model
  • Get the experiment definition/confiuguration in Python API, which you can use for automation or integration with other applications.

Saving the code for an experiment

To save an entire experiment as a notebook:

  1. After the experiment completes, click Save code from the Progress map pane.
  2. Assign a name and optional description and click Save.
  3. Click the link in the notification to open the notebook and review the code. You can also open the notebook from the Notebooks section of the Assets page of your project.

Saving an individual pipeline as a notebook

Save an individual pipeline as a notebook so you can review the Scikit-Learn source code for the trained model in a notebook.

Note: Currently, you can only generate a pipeline notebook for an AutoAI experiment that is created from a single data source and that is not a time series experiment.

What is included with the pipeline notebook

The experiment notebook provides annotated code that allow you to:

  • View the Scikit-learn pipeline definition
  • See the transformations applied for pipeline training
  • Review the pipeline evaluation

Saving a pipeline as a notebook

To save a pipeline as a notebook:

  1. Complete your AutoAI experiment.
  2. Select the pipeline you want to save in the leaderboard, and choose Save from the action menu for the pipeline, then Save as notebook.
  3. Name your notebook, add an optional description, and save it.
  4. Click the link in the notification to open the notebook and review the code. You can also open the notebook from the Notebooks section of the Assets page of your project.

Create sample notebooks

To see for yourself what AutoAI-generated notebooks look like:

  1. Follow the steps in AutoAI tutorial to create a binary classification experiment from sample data.
  2. After the experiment runs, click Save code in the experiment details panel.
  3. Name and save the experiment notebook.
  4. To save a pipeline as a model, choose a pipeline from the leaderboard, then click Save and Save as notebook.
  5. Name and save the pipeline notebook.
  6. From the Assets page of your project, open the resulting notebooks in the notebook editor and review the code.

Running an AutoAI notebook outside of Watson Studio

If your AutoAI notebook connects to a remote data source, for example a DB2 table, the connection is handled by a data connection service called Flight Server. To use the data connection service from outside of Watson Studio you must explicitly set the environment variables ‘FLIGHT_SERVICE_LOCATION’ and ‘FLIGHT_SERVICE_PORT’ to point to the location of the Flight Server. Ask your cluster administrator for the external route to the Flight Server for data connections.

Additional resources

  • For details on the methods used in the code, see Autoai-libs.
  • For more information on AutoAI notebooks, see this blog post.