Data Science

IBM Analytics Engine beta goes live

Share this post:

We are excited to announce the beta of the IBM Analytics Engine, providing a single Hadoop and Spark service under the Watson Data Platform.  It makes it easier for data engineers, data scientists and developers to develop and deploy analytics applications. With integration through Jupyter notebooks in Data Science Experience, IBM Analytics Engine provides the foundation for executing data science and machine learning workloads. The IBM Analytics Engine utilizes the Hortonworks Data Platform as the underlying Hadoop distribution, providing access to a market leading open source Hadoop distribution.

The IBM Analytics Engine provides the ability to spin up clusters within minutes, easily scale clusters up, and supports an external Hive metastores.  To create and manage cluster lifecycles, admins can use the Bluemix UI, REST APIs and the Cloud Foundry CLI. The latter two enables programmatic access to operationalize the use of Hadoop and Spark from external applications while deploying data pipelines. Jobs can also be submitted through a Cloud Foundry CLI extension providing a nice scriptable way to execute jobs remotely. Also key is the capability to pass scripts to customize clusters at creation time, which enables a predictable configuration across cluster creation and deletion cycles.

The architecture separates compute and storage for better scalability and reliability. This allows users to easily spin up clusters for the duration of a single job and delete them on completion. Users can execute jobs directly against data in the IBM Cloud Object Storage service and can make the analytic data even more resilient by using the cross-region option. Analytics Engine leverages Stocator when using Spark to improve data read and write speeds, thereby delivering better performance on I/O intensive workloads.

We believe that users should be focused on analyzing data instead of managing clusters and the intricacies of Hadoop or Spark platform configurations. We welcome your participation and feedback in the beta as we embark on simplifying the process and allow you to focus on gaining insights and taking action.

Available in the US-South region of the Bluemix catalog.

Offering Manager – AI OpenScale

More Data Science stories
May 2, 2019

Seamless Integration: Istio and External Services

By defining our own MCP server, we allow users to move to the Istio service mesh without any code and deployment model changes. This means we can easily use Istio to control, observe, connect, and secure services running outside Kubernetes clusters.

Continue reading

April 30, 2019

Introducing IBM Analytics Engine v1.2 and Announcing the Deprecation of IBM Analytics Engine v1.0

We are excited to inform you about the new version of IBM Analytics Engine v1.2 that will be available starting May 15, 2019. Along with this release, Analytics Engine v1.0 will be retired.

Continue reading

April 26, 2019

Help Shape the Future of Cloud Foundry

Are you a Cloud Foundry user? If so, here's your opportunity to influence the future of Cloud Foundry with the 2019 user survey.

Continue reading