GPUs

For faster execution of Spark applications, you can enable GPU scheduling and define the GPU mode on your hosts.

GPU overview
You can improve the performance of your Spark applications by offloading certain processing functions from your processor (CPU) to a graphics processing unit (GPU); specific hardware and software requirements apply.
Enabling GPUs
You can enable GPUs when you want to create instance groups to use GPU resources for applications and have GPU monitoring available.
Using resource groups with GPU hosts
Use dedicated resource groups for CPU hosts and GPU hosts. You can either create two new resource groups or use your existing resource groups. When you use existing resource groups, ensure that you specify one for CPU scheduling and the other for GPU scheduling.
Enabling adaptive scheduling
Optionally, enable adaptive scheduling for efficient usage of mixed resources (CPU and GPU) in your cluster. With adaptive scheduling, tasks run first on a portion of GPU resources in the cluster. When GPU resources are no longer available in the cluster, the remaining tasks run on CPU resources.
Creating an instance group to use GPUs
Create an instance group that allocates GPU slots to Spark executors in the instance group.
Submitting a Spark application with GPU RDD
You can submit an application to the instance group that uses GPU Resilient Distributed Dataset (RDD), which supports adaptive GPU scheduling.
Submitting a Spark application without GPU RDD
You can submit an application to the instance group to run jobs on GPUs.

Parent topic: Workload

Related tasks

Enabling a service to use GPUs