News
Abstract
Latest news and information for the IBM® packages for Apache Spark™.
Content
The following packages are available:
IBM Development Package for Apache Spark 
IBM z/OS® Package for Apache Spark
The latest news and information for the IBM z/OS Package for Apache Spark is provided in a Readme file on the package download page. See Downloads.
The remainder of this technote contains the latest news and information for the IBM Development Package for Apache Spark.
IBM Development Package for Apache Spark
These packages combine the data analytics capabilities of Apache Spark with IBM SDK, Java™ Technology Edition, Version 8. A number of packages for Linux are available to support different platform architectures. See Downloads.
Each package contains IBM SDK, Java Technology Edition, Version 8 and Apache Spark. To determine the release levels in each package, see Package build levels.
User documentation
Follow these links to find user documentation for each component in the package:
Supplementary information is contained in this technote.
Netty security issue (CVE-2016-4970)
IBM recommends that you do not configure and use netty-tcnative and the Netty OpenSslEngine with any version 1 release of the IBM Development Package for Apache Spark, due to CVE-2016-4970.
IBM recommends using version 2.1.0.0 and above of the package, which resolves this issue.
If you cannot update your installation, you should consider testing a configuration workaround that is published for this CVE. For more information, see the "Workarounds and Mitigations" section of the IBM Security Bulletin for CVE-2016-4970.
Bugs with 'Correctness' tag that are fixed in version 1.6.3
There are a number of bugs that have been resolved in the 1.6.3 release of Apache Spark.
IBM recommends updating to this version.
For more information, see the list of resolved issues in the Apache Spark JIRA.
Tungsten issue in a mixed Endian cluster
Spark SQL functions that use Tungsten optimizations are not currently supported in a mixed Endian cluster. For more information about the issue, including any updates, see Spark-12778.
Package build levels
Click on the build level links to see more information.
| Release date | IBM package | Java level | Apache Spark level |
| February 2017 | 1.6.3.1 | 8.0.4.1 | 1.6.3 |
| November 2016 | 1.6.3.0 | 8.0.3.20 | 1.6.3 |
| September 2016 | 1.6.2.1 | 8.0.3.10 | 1.6.2 |
| July 2016 | 1.6.2.0 | 8.0.3.0 | 1.6.2 |
| April 2016 | 1.6.1.1 | 8.0.3.0 | 1.6.1 |
| January 2016 | 1.6.1.0 | 8.0.2.10 | 1.6.1 |
| January 2016 | 1.6.0.1 | 8.0.2.10 | 1.6.0 |
| January 2016 | 1.6.0.0 | 8.0.2.0 | 1.6.0 |
| November 2015 | 1.5.2.0 | 8.0.2.0 | 1.5.2 |
[{"Product":{"code":"SSAHSS","label":"Development Package for Apache Spark"},"Business Unit":{"code":"BU053","label":"Cloud & Data Platform"},"Component":"General","Platform":[{"code":"PF016","label":"Linux"},{"code":"PF035","label":"z\/OS"}],"Version":"1.0","Edition":"All Editions","Line of Business":{"code":"","label":""}}]
Was this topic helpful?
Document Information
Modified date:
15 June 2018
UID
swg21974677