Updating instance groups to use updated components (Spark version, notebook packages, or Dask version)

When an updated component (Apache Spark version, notebook package, or Dask version) becomes available, upon adding them to your system to use with IBM® Spectrum Conductor, you can then update existing instance groups to use the upgraded components.

Before you begin

  • You must be a cluster administrator, consumer administrator, or have the Instance Groups Configure permission.
  • The components must be added to the system (see Adding Spark versions, Adding notebooks, or Adding Dask versions). The built-in Spark, notebook, and Dask versions are installed with IBM Spectrum Conductor. You are not required to add those packages, unless you removed them.
  • The instance group must be in the Registered, Ready, Register Error, or Deploy Error state. If the instance group is running workload, stop the instance group and all associated notebooks before you update it. See Stopping instance groups and Stopping notebooks in an instance group.

About this task

When an updated component package is added, that package becomes available for use with your instance groups. To enable an existing instance group to use the updated packages, that component package must be deployed to the instance group.

Procedure

  1. From the cluster management console, select Workload > Instance Groups.
  2. When component package updates are available for the instance group, make your selection as follows:
    Note: For updated notebook packages, the notebook is undeployed and the new version is deployed. Therefore, if you specified the notebook base data directory to be either in the same location or under the notebook's deployment directory, the base data directory is removed. To retain your data, manually back up the contents of the base data directory before you update the instance group configuration.
    • To only update the packages, under the Updates column, click the update for a instance group. When prompted, click Update.
    • To update the packages and change instance group settings, select the instance group to update and click Configure. When prompted, click Continue. For details on changing instance group settings, see Modifying instance groups.

Results

The updated packages are deployed to hosts of the instance group that use those packages.

What to do next

Start the instance group. See Starting instance groups.