What's new and changed in Execution Engine for Apache Hadoop

Installing or upgrading Execution Engine for Apache Hadoop

Ready to install or upgrade Execution Engine for Apache Hadoop?

Related documentation:

• Execution Engine for Apache Hadoop
• Installing Execution Engine for Apache Hadoop
• Upgrading Execution Engine for Apache Hadoop

Refresh 9 of Cloud Pak for Data Version 4.0

A new version of Execution Engine for Apache Hadoop was released in May 2022.

Operand version: 4.0.9

This release includes the following changes:

Security fixes

This release includes fixes for the following security issues:

CVE-2019-11250, CVE-2022-1154, CVE-2022-1271, CVE-2022-27191, CVE-2021-45346, CVE-2020-16598, CVE-2020-15778, CVE-2021-23841, CVE-2019-12904

Refresh 8 of Cloud Pak for Data Version 4.0

A new version of Execution Engine for Apache Hadoop was released in April 2022.

Operand version: 4.0.8

This release includes the following changes:

Bug fixes

This release includes the following fixes:

• Issue: Address high vulnerability CVE-2021-29923.

  Resolution: To address and resolve CVE-2021-29923, the Execution Engine for Apache Hadoop service for Cloud Pak for Data 4.0.8 or later requires the Execution Engine for Apache Hadoop RPM for 4.0.8 or later to be installed on the Hadoop cluster.

Security fixes

This release includes fixes for the following security issues:

CVE-2016-20012, CVE-2017-14166, CVE-2017-14501, CVE-2018-1000021, CVE-2018-1000654, CVE-2018-1000879, CVE-2018-1000880, CVE-2018-15919, CVE-2018-16428, CVE-2018-17360, CVE-2018-17794, CVE-2018-17985, CVE-2018-18309, CVE-2018-18483, CVE-2018-18484, CVE-2018-18605, CVE-2018-18606, CVE-2018-18607, CVE-2018-18700, CVE-2018-18701, CVE-2018-19211, CVE-2018-19217, CVE-2018-20002, CVE-2018-20623, CVE-2018-20651, CVE-2018-20657, CVE-2018-20673, CVE-2018-20839, CVE-2019-12904, CVE-2019-12972, CVE-2019-14250, CVE-2019-6110, CVE-2019-9071, CVE-2019-9074, CVE-2019-9075, CVE-2019-9077, CVE-2020-15778, CVE-2020-21674, CVE-2020-35512, CVE-2021-21300, CVE-2021-23177, CVE-2021-31566, CVE-2021-34798, CVE-2021-35937, CVE-2021-35938, CVE-2021-35939, CVE-2021-3634, CVE-2021-39275, CVE-2021-39537, CVE-2021-3997, CVE-2021-3999, CVE-2021-40330, CVE-2021-40528, CVE-2021-41617, CVE-2021-4209, CVE-2021-43618, CVE-2021-44568, CVE-2021-44569, CVE-2021-44570, CVE-2021-44571, CVE-2021-44573, CVE-2021-44574, CVE-2021-44575, CVE-2021-44576, CVE-2021-44577, CVE-2021-45960, CVE-2021-46143, CVE-2022-0261, CVE-2022-0318, CVE-2022-0359, CVE-2022-0361, CVE-2022-0392, CVE-2022-0413, CVE-2022-22822, CVE-2022-22823, CVE-2022-22824, CVE-2022-22825, CVE-2022-22826, CVE-2022-22827, CVE-2022-23218, CVE-2022-23219, CVE-2022-23308, CVE-2022-23852, CVE-2022-24329, CVE-2022-24975, CVE-2022-25235, CVE-2022-25236, CVE-2022-25313, CVE-2022-25314, CVE-2022-25315

Refresh 7 of Cloud Pak for Data Version 4.0

A new version of Execution Engine for Apache Hadoop was released in March 2022.

Operand version: 4.0.7

This release includes the following changes:

Security fixes

This release includes fixes for the following security issues:

CVE-2018-7208, CVE-2018-10373, CVE-2018-0494, CVE-2021-3487, CVE-2018-9996, CVE-2018-18314, CVE-2017-12883, CVE-2020-10878, CVE-2016-7098, CVE-2018-6798, CVE-2018-6797, CVE-2018-9138, CVE-2018-6543, CVE-2020-10543, CVE-2017-13089, CVE-2018-18311, CVE-2018-12015, CVE-2018-10535, CVE-2017-13090, CVE-2019-20916, CVE-2018-10372, CVE-2016-4971, CVE-2018-13033, CVE-2018-20483, CVE-2018-10534, CVE-2021-20197, CVE-2017-6508, CVE-2018-7568, CVE-2018-7569, CVE-2018-7642, CVE-2010-2252, CVE-2018-7643, CVE-2018-6759, CVE-2019-1010204, CVE-2017-12837, CVE-2018-7570, CVE-2018-18313, CVE-2009-3490, CVE-2020-12723, CVE-2022-21668, CVE-2021-20294, CVE-2018-6913, CVE-2021-37322, CVE-2018-8945, CVE-2019-5953, CVE-2018-19931, CVE-2018-18312, CVE-2014-4877, CVE-2021-34558, CVE-2009-0723, CVE-2009-0733, CVE-2021-38297, CVE-2021-36221, CVE-2021-43519, CVE-2016-20012, CVE-2009-0581, CVE-2021-39293

Refresh 6 of Cloud Pak for Data Version 4.0

A new version of Execution Engine for Apache Hadoop was released in February 2022.

Operand version: 4.0.6

This release includes the following changes:

New features

The 4.0.6 release of Execution Engine for Apache Hadoop includes the following features and updates:

Apache Livy 0.8.0 is now supported

When you use Spark 3.0 with Execution Engine for Apache Hadoop, you can run Apache Livy 0.8.0. When you run Livy 0.8.0, you can create Livy sessions with Python 3.8, which you could not do with Livy 0.7.0.

Bug fixes

This release includes the following fixes:

• Issue: The DSX-HI pod log no longer provides up-to-date logging of jobs submitted to Hadoop.

  Resolution: The issue is now resolved.

Security fixes

This release includes fixes for the following security issues:

CVE-2020-16135, CVE-2021-3521

Deprecated features

RStudio® Server with R 3.6, R notebooks, and Data Refinery jobs are no longer supported by Execution Engine for Apache Hadoop.

Refresh 5 of Cloud Pak for Data Version 4.0

A new version of Execution Engine for Apache Hadoop was released in January 2022.

Operand version: 4.0.5

This release includes the following changes:

Security fixes

This release includes fixes for the following security issues:

CVE 2021-45105, CVE 2021-45046, CVE-2021-44832, CVE-2021-38185, CVE-2017-15412, CVE-2018-12700, CVE-2021-3981, CVE-2020-1712, CVE-2017-0663, CVE-2017-9048, CVE-2020-27619, CVE-2017-18258, CVE-2017-9050, CVE-2021-3984, CVE-2021-4019, CVE-2017-9047, CVE-2019-16866, CVE-2018-1121, CVE-2018-20786, CVE-2021-3974, CVE-2016-5131, CVE-2017-9049, CVE-2021-39293, CVE-2021-3872, CVE-2016-4658, CVE-2019-20386, CVE-2021-3927, CVE-2017-7375

Refresh 4 of Cloud Pak for Data Version 4.0

A new version of Execution Engine for Apache Hadoop was released in December 2021.

Operand version: 4.0.4

This release includes the following changes:

New features

The 4.0.4 release of Execution Engine for Apache Hadoop includes the following features and updates:

Support for Power® 9 hardware

You can now install Execution Engine for Apache Hadoop on Red Hat® OpenShift® Container Platform Version 4.8 clusters running on Power 9 hardware.

Security fixes

This release includes fixes for the following security issues:

CVE-2021-20284, CVE-2021-3200, CVE-2021-28153, CVE-2021-33938, CVE-2020-10001, CVE-2019-13750, CVE-2021-33503, CVE-2021-23840, CVE-2021-25317, CVE-2021-22898, CVE-2020-15257, CVE-2021-3778, CVE-2019-17594, CVE-2020-13790, CVE-2021-29921, CVE-2020-12401, CVE-2019-20838, CVE-2021-21334, CVE-2021-35942, CVE-2021-3572, CVE-2019-2201, CVE-2020-14145, CVE-2020-17541, CVE-2021-33574, CVE-2020-24370, CVE-2021-33197, CVE-2020-35448, CVE-2021-42574, CVE-2021-3468, CVE-2020-14155, CVE-2021-3426, CVE-2019-13751, CVE-2021-27645, CVE-2019-18218, CVE-2021-36085, CVE-2021-20232, CVE-2021-22925, CVE-2021-23336, CVE-2021-36087, CVE-2019-19603, CVE-2020-13435, CVE-2021-22876, CVE-2020-12399, CVE-2021-33928, CVE-2019-5827, CVE-2020-12762, CVE-2021-20231, CVE-2021-3796, CVE-2021-36086, CVE-2021-33929, CVE-2021-3445, CVE-2019-17595, CVE-2017-6519, CVE-2018-16429, CVE-2021-36084, CVE-2021-20266, CVE-2021-33930, CVE-2017-18018, CVE-2020-12413, CVE-2021-33560

Refresh 3 of Cloud Pak for Data Version 4.0

A new version of Execution Engine for Apache Hadoop was released in November 2021.

Operand version: 4.0.3

This release includes the following changes:

Security fixes

This release includes fixes for the following security issues:

CVE-2008-5349, CVE-2021-22924, CVE-2020-25648, CVE-2021-22946, CVE-2008-3110, CVE-2008-5352, CVE-2021-22923, CVE-2008-5358, CVE-2008-3109, CVE-2021-22947, CVE-2008-3105, CVE-2021-22922, CVE-2008-5347, CVE-2008-3106, CVE-2008-1191, CVE-2008-3103, CVE-2021-36222, CVE-2007-3716, CVE-2021-37750

Refresh 2 of Cloud Pak for Data Version 4.0

A new version of Execution Engine for Apache Hadoop was released in October 2021.

Operand version: 4.0.2

This release includes the following changes:

New features

The 4.0.2 release of Execution Engine for Apache Hadoop includes the following features and updates:

Support on s390x hardware

You can train, and deploy machine learning models on Cloud Pak for Data on Red Hat OpenShift Container Platform Version 4.8 running on IBM Z® and LinuxONE. However, deployments on this hardware support a limited set of features.

For a list of the features that are available on IBM Z and LinuxONE, see Capabilities on IBM Z.

Security fixes

This release includes fixes for the following security issues:

CVE-2020-8231, CVE-2020-8927, CVE-2020-13776, CVE-2021-27219, CVE-2020-9951, CVE-2019-2708, CVE-2021-2341, CVE-2020-29361, CVE-2020-8286, CVE-2021-20271, CVE-2021-3517, CVE-2020-24332, CVE-2020-26137, CVE-2020-27619, CVE-2021-3518, CVE-2020-25658, CVE-2020-13543, CVE-2020-29363, CVE-2016-10228, CVE-2019-3842, CVE-2020-27618, CVE-2021-31879, CVE-2020-24331, CVE-2020-27783, CVE-2020-15358, CVE-2021-20191, CVE-2021-27218, CVE-2019-18276, CVE-2021-3421, CVE-2020-24977, CVE-2020-13434, CVE-2019-13012, CVE-2021-3326, CVE-2021-23336, CVE-2020-8284, CVE-2021-3537, CVE-2017-14502, CVE-2019-9169, CVE-2020-8285, CVE-2020-9983, CVE-2020-28196, CVE-2019-25013, CVE-2020-14343, CVE-2021-2369, CVE-2020-13584, CVE-2021-3177, CVE-2021-3520, CVE-2020-26116, CVE-2020-24330, CVE-2021-3541, CVE-2021-3516, CVE-2020-29362, CVE-2021-2388, CVE-2021-33910, CVE-2020-9948

Refresh 1 of Cloud Pak for Data Version 4.0

A new version of Execution Engine for Apache Hadoop was released in August 2021.

Operand version: 4.0.1

This release includes the following changes:

New features

The 4.0.1 release of Execution Engine for Apache Hadoop includes the following features and updates:

Support for upgrade

You can now upgrade Execution Engine for Apache Hadoop from the following Cloud Pak for Data releases:

• Cloud Pak for Data Version 3.5.x
• Cloud Pak for Data Version 4.0.x

Security fixes

This release includes fixes for the following security issues:

CVE-2020-15945

Initial release of Cloud Pak for Data Version 4.0

A new version of Execution Engine for Apache Hadoop was released as part of Cloud Pak for Data Version 4.0.

Operand version: 4.0.0

This release includes the following changes:

New features

Version 4.0.0 of the Execution Engine for Apache Hadoop service is available on Cloud Pak for Data 4.0. Execution Engine for Apache Hadoop no longer requires a separate license to install. It is included in your purchase of Cloud Pak for Data. This release of Execution Engine for Apache Hadoop does not include new features or updates.