Mainframe data becomes more accessible with Apache Spark

Share this post:

The rise of digital business has led to a proliferation of data types and sources – from location data to social media content – stored in disparate formats in an enterprise’s data centers or the public cloud. This data can be extremely valuable to organizations that want to engage their customers in real-time in new, more personalized ways to improve customer service and generate more revenue.

The problem is, companies either can’t access this data, or it takes them hours to days to do so, by which point it’s no longer as relevant or useful. In the Cognitive Era, where data is the new natural resource and computer systems are able to understand, reason and learn, organizations need fast, more secure access to a wide range of data, including the rich data sets on the IBM z Systems mainframe, running z/O 2.1 or later to better serve customers and beat their competition. They need instant analysis of this data to act on market intelligence, instead of simply reacting to market trends.

This week at Strata + Hadoop World in San Jose, IBM unveiled the z/OS Platform for Apache Spark. The platform allows developers and data scientists to access and analyze data in-place on z Systems with the Apache Spark open-source analytics framework – potentially saving their organizations time and money and limiting risk. Previously, they would have had to move data off the platform in batches and analyze it that way — a process that would take hours to days.

z Systems handles critical data and transactions for many of the world’s major banks, insurers, retailers and transport companies. It features the industry’s fastest commercial microprocessor and can score predictive models within a transaction in 2 milliseconds or less.[1] z Systems also meets the highest levels of security to help data remain protected.[2]

To further advance the new capabilities, we’ve also partnered with DataFactz, Zementis and Rocket Software to create customized solutions using IBM z/OS Platform for Apache Spark:

  • DataFactZ is a new partner that is working with IBM to develop Spark analytics based on Spark SQL and MLlib for data and transactions processed on the mainframe.
  • Rocket Software has been a long-standing IBM partner and this collaboration extends to z/OS Apache Spark. For example, the new Rocket Launchpad solution will allow clients to try the platform using data on z/OS.
  • Zementis is complementing its in-transaction predictive analytics offering for z/OS with a standards-based execution engine for Apache Spark. The solution helps allow users to deploy and execute advanced predictive models that can help them anticipate end users’ needs, compute risk or detect fraud in real-time at the point of greatest impact, while processing a transaction.

By enabling Spark natively on the mainframe, clients will be able to take even greater advantage of z Systems’ world-class capabilities of speed, scale and security as they transform into cognitive businesses.

You can find more details on the new z/OS Platform for Apache Spark in this overview, which includes a use case demo of what the new offering can do for an organization.

[1] World’s fastest microprocessor is based on 5.5GHz core processor speed; Based on IBM tests of I/O processing on the z13s. that show results in 2 milliseconds or less.

[2] Based on Common Criteria EAL5+ security rating for z Systems mainframe

More stories

The unbearable lightness of being: mainframe on the cloud

Cloud computing, Mainframes, Servers

Some time ago, having the words “cloud” and “mainframe” in the same phrase was considered implausible or even impossible. Working with mainframe-related technologies or even directly with any mainframe applications was always associated with green screens, blocky letters and this feeling of old technology. We could say this feeling still exists among a lot of more

The next “tick” of SUSE Linux Enterprise Server for IBM Z and LinuxONE

Linux on z, LinuxONE solutions, Mainframes

The working philosophy at Fort Vale is “continuous improvement” according to the UK-based engineering company deploying its new ERP system on SUSE Linux Enterprise Server and IBM LinuxONE. Continuous improvement is also a good theme for the next “tick” of SUSE Linux Enterprise Server – following the “tick-tock” approach that SUSE uses for releases. SLES more

IBM Z enhances the journey to cloud

Cloud computing, Linux on z, Mainframes

In today’s fast-paced world, forward-thinking enterprises constantly seek to innovate while continuing to delight their customers. Cloud–public, private and hybrid–underpins how these enterprises are innovating while also introducing new challenges. The world is not getting any simpler. It’s not moving to fewer clouds. It’s not moving to fewer options. Hybrid multicloud is the new standard. more