Spark Environments Replace the Deprecated Apache Spark as a Service
5 min read
Support for Apache Spark as a Service integration with Watson Machine Learning and Watson Studio is being deprecated.
This integration will be retired on June 28, 2019. We are happy to announce that this service integration has been replaced by built-in Spark environments.
Benefits of Spark environments
- Spark kernels on-demand: Save time and energy to focus your analysis; create a Spark environment in Watson Studio and launch directly into a notebook.
- Configurable, elastic compute: Configure your Spark environment and choose your kernel hardware configurations from Watson Studio.
- Easily share your environment : Your collaborators can easily use Spark environments.
- Multiple language support :— Choose from the most popular languages for your Spark kernels (Python 3, R, Scala).
Switching to built-in Spark environments
If you currently use the Apache Spark as a Service in any of the following ways, you must switch to using built-in Spark environments:
- Batch Deployments
- Model Builder
- Modeler Flows with Spark Runtime
Here are the dates you need to know
Service Retirement Announce Date: June 4, 2019
End of Support Date: June 28, 2019
For a period of 24 days after the Service Retirement Announce Date, through June 28, 2019, all existing Spark as a Service integrations with Watson Machine Learning and with Watson Studio projects will continue to be supported. After June 28, 2019, Spark as a Service will no longer be available to Watson Machine Learning or by tools in Watson Studio like Notebooks and Model Builder. Spark-powered realtime streaming predictions will no longer be available.
We strongly recommend you update as soon as possible to work with the newer, more flexible Spark engines in Watson Machine Learning and Watson Studio.