HDFS Transparency support matrix
The support matrix for HDFS Transparency and the IBM Storage® Scale Big Data Analytics Integration Toolkit for HDFS Transparency (Toolkit for HDFS) is bundled together to work for the supported IBM Storage Scale release.
| IBM Storage Scale version | HDFS Transparency version | ||
|---|---|---|---|
| 3.1.1-x stream | 3.2.2-x stream | 3.3.x-y stream | |
| 6.0.0.2 | N/A | N/A | 3.3.6-3 |
| 6.0.0.1 | N/A | N/A | 3.3.6-2 |
| 6.0.0.0 | N/A | N/A | 3.3.6-1 |
| 5.2.3.7 | 3.1.1-20 | 3.2.2-13 | 3.3.6-4 |
| 5.2.3.6 | 3.1.1-20 | 3.2.2-13 | 3.3.6-3 |
| 5.2.3.5 | 3.1.1-19 | 3.2.2-12 | 3.3.6-1 |
| 5.2.3.4 | 3.1.1-19 | 3.2.2-11 | 3.3.6-1 |
| 5.2.3.3 | 3.1.1-19 | 3.2.2-11 | 3.3.6-0 |
| 5.2.3.0 or 5.2.3.2 | 3.1.1-18 | 3.2.2-11 | 3.3.6.0 |
| 5.2.2.0 - 5.2.2.1 | 3.1.1-18 | 3.2.2-9 | N/A |
| 5.2.1.1 | 3.1.1-18 | 3.2.2-9 | N/A |
| 5.2.1.0 | 3.1.1-17 | 3.2.2-9 | N/A |
| 5.2.0.1 | 3.1.1-17 | 3.2.2-9 | N/A |
| 5.2.0.0 | 3.1.1-17 | 3.2.2-8 | N/A |
| 5.1.9.12 | 3.1.1.19 | 3.2.2-12 | N/A |
| HDFS Transparency version | Tested Hadoop distribution1 | Tested Hadoop client version | OS | Platform | JAVA OpenJDK version |
|---|---|---|---|---|---|
| 3.1.1-x | See CDP support matrix | Hadoop 3.1.1 |
RHEL 8.x |
x86_64 ppc64le |
OpenJDK 8 |
| N/A | Hadoop 3.1.1 |
RHEL 8.x |
|||
| 3.2.2-x | Apache BigTop 3.2.1 | Hadoop 3.2.2 |
RHEL 8.x |
x86_64 |
OpenJDK 8 |
| N/A | Hadoop 3.2.2 |
RHEL 8.x |
|||
| 3.3.6-x | Apache BigTop 3.2.1 | Hadoop 3.2.2 |
RHEL 8.x |
x86_64 |
OpenJDK 8 |
| Apache BigTop 3.3.0 | Hadoop 3.3.6 |
RHEL 8.x |
|||
| N/A | Hadoop 3.3.6 |
RHEL 8.x |
|||
| N/A | Hadoop 2.10.2 CLI (standalone) |
RHEL 8.x RHEL 9.x |
x86_64 |
OpenJDK 8 |
|
|
1 Although a list of tested Hadoop stacks is provided, HDFS Transparency is an HDFS-compliant file system and can work with other Hadoop stacks. The Cloudera Distribution for Hadoop (CDP) tested Hadoop stacks are recommended configurations for use with HDFS Transparency. IBM supports HDFS Transparency when the operating system, architecture, and Java version are supported. IBM recommends validating the environment in a test system before deploying to production. IBM does not support Hadoop stacks and services that operate above HDFS Transparency. 2 RHEL 10.x is supported for HDFS Transparency and was tested across all HDFS Transparency versions for HDFS client interfaces. Apache BigTop and CDP did not support RHEL 10.x at the time of testing. Therefore, full distribution stack testing was not performed. |
|||||
- Unlike previous versions of HDFS Transparency, HDFS Transparency 3.1.1-x, 3.2.2-x, and 3.3.x-y are tightly coupled with IBM Storage Scale. You need to upgrade the IBM Storage Scale package to get the correct supported versions for CES HDFS.
- If the OS is not supported for a specific IBM Storage Scale release, then it is also not supported for HDFS Transparency.
- Unlike previous versions of HDFS Transparency 3.1.1-x, HDFS Transparency 3.1.1-15+ is delivered without dependent JAR files. For more information about the installation process, see Installation prerequisites.
- Unlike previous versions of HDFS Transparency 3.2.2-x, HDFS Transparency 3.2.2-6+ is delivered without dependent JAR files. For more information about the installation process, see Installation prerequisites.