IBM Support

IBM packages for Apache Spark: Latest news and information



Latest news and information for the IBM® packages for Apache Spark™.


The following packages are available:

  • Linux platforms IBM Development Package for Apache Spark
  • z/OS platforms IBM z/OS® Package for Apache Spark

The latest news and information for the IBM z/OS Package for Apache Spark is provided in a Readme file on the package download page. See Downloads.

The remainder of this technote contains the latest news and information for the IBM Development Package for Apache Spark.

IBM Development Package for Apache Spark

These packages combine the data analytics capabilities of Apache Spark with IBM SDK, Java™ Technology Edition, Version 8. A number of packages for Linux are available to support different platform architectures. See Downloads.

Each package contains IBM SDK, Java Technology Edition, Version 8 and Apache Spark. To determine the release levels in each package, see Package build levels.

User documentation

Follow these links to find user documentation for each component in the package:

Supplementary information is contained in this technote.

Netty security issue (CVE-2016-4970)

IBM recommends that you do not configure and use netty-tcnative and the Netty OpenSslEngine with any version 1 release of the IBM Development Package for Apache Spark, due to CVE-2016-4970.

IBM recommends using version and above of the package, which resolves this issue.

If you cannot update your installation, you should consider testing a configuration workaround that is published for this CVE. For more information, see the "Workarounds and Mitigations" section of the IBM Security Bulletin for CVE-2016-4970.

Bugs with 'Correctness' tag that are fixed in version 1.6.3

There are a number of bugs that have been resolved in the 1.6.3 release of Apache Spark.
IBM recommends updating to this version.

For more information, see the list of resolved issues in the Apache Spark JIRA.

Tungsten issue in a mixed Endian cluster

Spark SQL functions that use Tungsten optimizations are not currently supported in a mixed Endian cluster. For more information about the issue, including any updates, see Spark-12778.

Package build levels
Click on the build level links to see more information.

Release date IBM packageJava level Apache Spark level
February 2017 1.6.3
November 2016 1.6.3
September 2016 1.6.2
July 2016 1.6.2
April 2016 1.6.1
January 2016 1.6.1
January 2016 1.6.0
January 2016 1.6.0
November 2015 1.5.2

Go back to top

[{"Product":{"code":"SSAHSS","label":"Development Package for Apache Spark"},"Business Unit":{"code":"BU053","label":"Cloud & Data Platform"},"Component":"General","Platform":[{"code":"PF016","label":"Linux"},{"code":"PF035","label":"z\/OS"}],"Version":"1.0","Edition":"All Editions","Line of Business":{"code":"","label":""}}]

Document Information

Modified date:
15 June 2018