Jupyter notebooks

IBM® Spectrum Conductor bundles Jupyter notebooks to provide an interactive environment for data manipulation and visualization.

Spark-ready kernels

IBM Spectrum Conductor provides support for the following Spark-ready kernels:

Python
Scala
R

Important: To use the R kernel, you must set up the R environment and preinstall all of the required R packages on all driver resource groups. During instance group creation, you must also set the spark.r.command parameter with the full path to your R script location.

Jupyter Enterprise Gateway

IBM Spectrum Conductor works with the Jupyter Enterprise Gateway, which is a web server that launches kernels on remote servers throughout the enterprise. The Enterprise Gateway launcher includes the -RemoteProcessProxy.spark-context-initialization-mode option, which indicates the time frame in which the Spark context is created. IBM Spectrum Conductor supports the eager mode, which attempts to create the Spark context as soon as possible.

Home directory for notebook workload

Some functions of the Jupyter notebook require that the notebook workload execution user have an existing home directory. Ensure this home directory exists before running Jupyter notebook workload.