Spark workload
In this module, you learn how to manage Spark workload using IBM® Spectrum Conductor.
As part of managing Spark workload, you can add Spark versions and notebooks; then, create and manage Spark instance groups. Once you create a Spark instance group, you can submit Spark applications to the instance group and manage those applications.
Learning objectives
After completing the lessons in this module, you will understand the concepts and know how to do the following tasks:- Create, deploy and start a Spark instance group
- Create a template from the Spark instance group
- Create and share a notebook
- Submit and monitor a Spark batch application
- Schedule a Spark batch application
- Open the notebook and create a new note
- Monitor and modify the Spark instance group to change a Spark configuration
- Configure the Spark instance group to run within Docker containers
Time required
This module should take approximately 45 minutes to complete.- Create a Spark instance group
In this lesson, you create a Spark instance group. - Create template from the Spark instance group
In this lesson, you create a template from the Spark instance group you previously created. - Create and share a notebook
In this lesson, you create a notebook for the Spark instance group and share the notebook with multiple users known as collaborators. - Submit and monitor a Spark batch application
In this lesson, you submit and monitor a Spark batch application. - Schedule a Spark batch application
In this lesson, you schedule a Spark batch application. - Open a notebook and create a note
In this lesson, you open a notebook and create a note. - Monitor and modify a Spark instance group
In this lesson, you learn how to monitor and modify the sample Spark instance group. - Configure the Spark instance group to use data connectors
In this lesson, you configure the sample Spark instance group to use data connectors from the Data Connectors tab. - Configure the Spark instance group to use Docker
In this lesson, you configure the sample Spark instance group to run within Docker containers.