What's New

Introducing IBM Analytics for Apache Spark

Share this post:

A few weeks ago at the Spark Summit 2015, IBM announced an unprecedented commitment to Apache Spark with a series of investments designed to advance innovation through the open source community, and accelerate Spark adoption across developers and data scientists. These investments demonstrate IBM’s commitment to open source technologies, and together are one of the largest ventures our company has made in the open source arena since Linux.

The highlight of IBM’s announcement is the introduction of Apache Spark as a service, IBM Analytics for Apache Spark, available in IBM Bluemix, IBM’s platform for cloud services. IBM views Spark as a transformational technology, with powerful new capabilities that have the potential to materially change the way businesses manage thru digital transformation. On behalf of the IBM Analytics Apache for Spark team, we are all very excited to make that power accessible and useful to you.

IBM Analytics for Apache Spark will enable powerful analytics without the hassle of setup, maintenance and operations

My interest in bringing this product to life is deeply personal. As a prior VP of Analytics at a public cable company, I lived the challenge on a day-to-day basis of generating meaningful insights from highly disparate data. While I had a great team, they each used very different tools and thus were highly siloed by default. Having several specialized tools did not mean that they were efficient – simple analysis and changes required great efforts regardless of the specific tool. For those reasons, I’m very excited about the potential of Spark and its ability to truly transform businesses, and to do so in a common platform that can be used by anyone from an entry level analyst to a data scientist to a developer. Looking back, I realize today that we are finally in a place that I could only imagine several years back. I’m helping build the tool I wish I had had, and am personally very gratified to see IBM’s vision come to life.

IBM Analytics for Apache Spark

IBM will soon introduce a public beta of IBM Analytics for Apache Spark for developers, business analysts, and data scientists to embrace and take advantage of Apache Spark at no cost. This exciting new service includes:

  • 100% open source Spark implementation operating the latest official release
  • An integrated Notebook service based on the open source Jupyter project to support interactive and reproducible analytics in Scala and Python
  • Connectors for common data sources, starting with SWIFT Object Storage, with additional support for Cloudant, dashDB, SQLDB and other IBM Cloud Data Services and standards coming soon.

IBM Analytics for Apache Spark’s unified programming model for batch and real-time analytics, multi-language support, in-memory data processing engine, and integrated libraries such as machine learning and graph processing, make it possible for developers, business analysts, and data scientists to incorporate intelligence into every application and better support critical-path business decision making.

The service will allow any developer, business analyst, or data scientist to explore, experiment, and develop powerful analytics and new applications without the hassle of setup, maintenance and operations, which often holds back innovation. Additionally, the service includes freedom to later bring Spark in-house or move to another provider. Our initial offering is only the start – we intend to add new features, new connectors and to provide access to new datasets as we continue developing the service.

Register for the beta!

As a proud member of the team bringing you the IBM Analytics for Apache Spark service, I invite you to register for the beta at ibm.co/getspark. Our only ask is to let us know how we can do even better in making Spark easier to consume and leverage to transform your business and apps.

To find more information about IBM’s commitment to Spark, including information on the new Spark Technology Center, visit ibm.com/spark. To pre-register for IBM Analytics for Apache Spark, visit ibm.co/getspark.

More stories
April 30, 2019

Introducing IBM Analytics Engine v1.2 and Announcing the Deprecation of IBM Analytics Engine v1.0

We are excited to inform you about the new version of IBM Analytics Engine v1.2 that will be available starting May 15, 2019. Along with this release, Analytics Engine v1.0 will be retired.

Continue reading

April 23, 2019

Announcing the Deprecation of the Watson Machine Learning JSON Token Authentication Service

We’d like to inform you about the deprecation of the Watson Machine Learning JSON Token Authentication service. This method of authentication will be retired on May 30, 2019.

Continue reading

April 19, 2019

Introducing IBM Cloud Object Storage Firewall: Further Secure Your Data

IBM Cloud Object Storage (COS) is giving you more control over who can access your data. We have introduced a new capability allowing you to configure your buckets with trusted IP address(es) that will dictate access to the data in COS.

Continue reading