Share this post:
If you are wondering whether Spark is “for real” as an enterprise solution, the Spark Summit 2015, held recently in San Francisco, strongly suggested that we will be seeing a lot more of it in enterprise environments in the coming months. A major talking point was the commitment IBM showed to Spark at this event. In addition to IBM, there were speakers from several large organizations including NASA/JPL, NBC, Netflix, Beth Israel Deaconess Medical Center, Capital One, Thomson Reuters, the CIA, Toyota, and the US Patent & Trademark Office.
The Spark Summit itself is certainly growing. With over 2000 attendees, the Summit was the biggest by far to date and testimony to Spark’s growing momentum. The plenary sessions were typically full to capacity, and three parallel breakout tracks — developer, data science, and applications and use cases – were also well attended. Likewise, the solution expo often appeared to be bursting at the seams and the attendance was probably as large as the venue could handle.
IBM was a platinum sponsor of the event. Beth Smith, General Manager, Analytics Platform, IBM Analytics, gave a keynote address in which she made several important announcements. These included plans to:
- Donate the IBM SystemML machine learning technology to the Spark open source ecosystem
- Educate more than one million data scientists and data engineers on Spark
- Establish a Spark Technology Center in San Francisco to foster innovation
Copies of all Spark Summit presentation materials, along with video, are available from the Spark Summit 2015 website.
On the first evening of the Summit, IBM hosted a Spark Community Event. While guests enjoyed cocktails and snacks, they were able to view several IBM Spark demos. One of these showed a complex genomics sequencing workflow running on an IBM Software Defined Infrastructure platform including IBM Platform LSF, IBM Spectrum Scale, IBM Platform Symphony and, to integrate Spark into the workflow, IBM Platform Application Service Controller.
The centerpiece of the Community Event was a lively hour-long panel discussion of Spark and its impact on industry and education. The end-user customer voice on this panel was USAA, a Platform Symphony customer. The USAA representative was Robert Dohner, an Associate Actuary who discussed the role of Spark in his organization. The panel was moderated by Edd Dumbill, VP, Strategy & Marketing at Silicon Valley Data Science and included speakers from education and industry.
The overall message of the whole event was probably best summarized by a memorable line from entrepreneur and venture capitalist Ben Horwitz of Andreessen Horowitz. Speaking of the impact that IBM’s announcements would have on Spark in the enterprise environment, Ben quipped “Think of IBM as a rabbi” who has pronounced that Spark “is kosher for use in your company… Spark is now blessed by the enterprise rabbi.”
For more information about how IBM Software Defined Infrastructure solutions can help with Spark deployment for the enterprise, please visit the IBM Platform Application Service Controller web page.