HDFS Transparency support matrix
The support matrix for HDFS Transparency and the IBM Storage® Scale Big Data Analytics Integration Toolkit for HDFS Transparency (Toolkit for HDFS) is bundled together to work for the supported IBM Storage Scale release.
| IBM Storage Scale version | HDFS Transparency version | ||
|---|---|---|---|
| 3.1.1-x stream | 3.2.2-x stream | 3.3.x-y stream | |
| 6.0.0.1 | N/A | N/A | 3.3.6-2 |
| 6.0.0.0 | N/A | N/A | 3.3.6-1 |
| 5.2.3.5 | 3.1.1-19 | 3.2.2-12 | 3.3.6-1 |
| 5.2.3.4 | 3.1.1-19 | 3.2.2-11 | 3.3.6-1 |
| 5.2.3.3 | 3.1.1-19 | 3.2.2-11 | 3.3.6-0 |
| 5.2.3.0 or 5.2.3.2 | 3.1.1-18 | 3.2.2-11 | 3.3.6.0 |
| 5.2.2.0 - 5.2.2.1 | 3.1.1-18 | 3.2.2-9 | N/A |
| 5.2.1.1 | 3.1.1-18 | 3.2.2-9 | N/A |
| 5.2.1.0 | 3.1.1-17 | 3.2.2-9 | N/A |
| 5.2.0.1 | 3.1.1-17 | 3.2.2-9 | N/A |
| 5.2.0.0 | 3.1.1-17 | 3.2.2-8 | N/A |
| 5.1.9.12 | 3.1.1.19 | 3.2.2-12 | N/A |
| HDFS Transparency version | Tested Hadoop distribution1 | Tested Hadoop client version | OS | Platform | JAVA OpenJDK version |
|---|---|---|---|---|---|
| 3.1.1-x | See CDP support matrix | Hadoop 3.1.1 |
RHEL 8.x |
x86_64
ppc64le |
OpenJDK 8 |
|
3.2.2-x |
Apache BigTop 3.2.1 | Hadoop 3.2.2 |
RHEL 8.x |
x86_64 |
OpenJDK 8 |
| 3.3.6-x | Apache BigTop 3.2.1 | Hadoop 3.2.2 |
RHEL 8.x |
x86_64 |
OpenJDK 8 |
| Apache BigTop 3.3.0 | Hadoop 3.3.6 |
RHEL 8.x |
x86_64 |
OpenJDK 8 |
|
| CDP 7.1.9 | Hadoop 3.1.1 |
RHEL 8.x |
x86_64 |
OpenJDK 8 |
|
| N/A | Hadoop 2.10.2 CLI (standalone) |
RHEL 8.x |
x86_64 |
OpenJDK 8 |
|
|
1 Although a list of tested Hadoop stacks is provided, HDFS Transparency is an HDFS-compliant file system and can work with other Hadoop stacks. The tested Hadoop stacks are recommended configurations for use with HDFS transparency. IBM supports HDFS Transparency when the operating system, architecture, and Java version are supported. IBM recommends validating the environment in a test system before deploying to production. It is not recommended to use Hadoop stacks other than those listed without extensive Hadoop experience or support from the respective stack vendor. IBM does not support the selected Hadoop stack and its services, which operate above HDFS Transparency. Support for the Hadoop stack and its services is provided by the respective stack vendor or requires self-support. |
|||||
- Unlike previous versions of HDFS Transparency, HDFS Transparency 3.1.1-x, 3.2.2-x, and 3.3.x-y are tightly coupled with IBM Storage Scale. You need to upgrade the IBM Storage Scale package to get the correct supported versions for CES HDFS.
- If the OS is not supported for a specific IBM Storage Scale release, then it is also not supported for HDFS Transparency.
- Unlike previous versions of HDFS Transparency 3.1.1-x, HDFS Transparency 3.1.1-15+ is delivered without dependent JAR files. For more information about the installation process, see Installation prerequisites.
- Unlike previous versions of HDFS Transparency 3.2.2-x, HDFS Transparency 3.2.2-6+ is delivered without dependent JAR files. For more information about the installation process, see Installation prerequisites.