Execution Engine for Apache Hadoop on Cloud Pak for Data

The Execution Engine for Apache Hadoop services integrates the Watson Studio service with your remote Apache Hadoop cluster.

Data scientists can use this service for the following tasks:

• Browse remote Hadoop data through connections
• Cleanse and shape remote Hadoop data with Data Refinery
• Run Data Refinery and jobs on the Hadoop Spark cluster
• Run a notebook session on the remote Hadoop system
• Access Hadoop systems with basic utilities from RStudio and Jupyter notebooks

The Execution Engine for Apache Hadoop includes:

• Services that establish secure connections between Watson Studio and Hadoop
• Integration with Hadoop for Refinery and Notebook
• A high availability configuration to the remote Hadoop system
• Utilities that connect Watson Studio and Hadoop

The service requires a service user who has the necessary privileges to submit requests on behalf of the Watson Studio users to WebHDFS, WebHCAT, Spark, and YARN. The service generates a secure URL for each Watson Studio cluster that is integrated with the Hadoop cluster.