Set up runtime environments

To avoid slow compute performance within your project runtime environments, you can reserve CPU and memory resources in advance from the Environments page in your project.

Important: Reserving CPU and memory reduces the availability of CPU and memory for other runtime environments. However, if you disable Reserve CPU cores or Reserve memory you can impact system performance if the available CPU or RAM on the servers are over-committed.

A runtime environment represents an allocation of compute resources (one or more Docker containers) on the cluster. You can define multiple environments for specific images, such as RStudio and notebooks.

  • To change the CPU or memory allocations for a specific runtime environment, click the name of the environment and adjust the number of CPU cores or Memory that is allocated to it. Then click Save and restart.
  • To stop an environment, click the Stop icon (Stop) on the environment tile.
  • To see all files, processes, and clusters related to the environment, click the Info icon (Info).
  • If you installed a library or package from a Jupyter notebook or terminal, for example, conda install -y arrow, you can the library or package in a custom image on the cluster by clicking the Save icon (Save). The custom image then appears in the new My Images panel. See Manage IBM Watson Studio packages as a user for details.

Managing resources

If there are insufficient resources to run a job, you can view all runtime environments in the cluster to determine whether any of the active environments can be stopped to free up resources. To view the list of runtime environments, click All Active Environments page from the application menu.

From the All Active Environments page, you can view and sort the users who created each environment and determine how much CPU and memory each environment is using and how much of each resource each environment has reserved (unless the environment uses unmanaged resources).

To stop a running environment, worker, or service, click Stop now next to it.

Important: Users do not receive a notification that their environment has been stopped.

Troubleshooting runtime environments

Use the following recommendations if you encounter an issue with a runtime environment:
  • If a notebook contains a data frame that loads a large data file and you receive an error that the kernel died, you can edit the runtime environment to increase the amount of memory.
  • If a notebook crashes or the spark context becomes unavailable (indicated by sc undefined in the notebook):
    1. Click Stop next to the environment.
    2. Return to the notebook to get the spark context running again.
  • If you allocate the maximum amount of CPU for a runtime environment, the environment might stay in Pending state indefinitely. To resolve the issue, reduce the CPU that is allocated to the environment.