Spark versions

You can download and import new Spark versions for use with IBM® Spectrum Conductor. These Spark versions are prepackaged to include Apache Spark binaries and other files required for IBM Spectrum Conductor capabilities.

While the Spark version packages that are included with IBM Spectrum Conductor are integrated with Apache Spark, they are not identical. The packages includes several enhancements to enable Spark to run with the resource orchestrator for improved application performance and more efficient resource usage. Additionally, the packages includes scripts to enable automatic deployment and lifecycle management of the instance group.