IBM Storage Scale support for Hadoop

IBM Storage Scale provides integration with Hadoop applications that use the Hadoop connector.

If you plan to use a Hadoop distribution with the Hadoop connector, see the chapter that corresponds to your Cloudera distribution (CDP Private Cloud Base) or the Apache Hadoop under the big data and analytics support documentation.

Different Hadoop connectors

• Second generation HDFS Transparency
  • IBM Storage Scale HDFS Transparency (also known as, HDFS Protocol) offers a set of interfaces that allows applications to use HDFS client to access IBM Storage Scale through HDFS RPC requests. HDFS Transparency implementation integrates both the NameNode and the DataNode services and responds to the request as if it were HDFS.
• First generation Hadoop connector
  • The IBM Storage Scale Hadoop connector implements Hadoop file system APIs and the FileContext class so that it can access the IBM Storage Scale.