Big Data File stage

You can use the Big Data File stage to access files on the Hadoop Distributed File System (HDFS). You use the Big Data File stage to read and write HDFS files.

Overview

The Big Data File stage is similar in function to the Sequential File stage. You can use the stage to process multiple files and preserve the multiple files on the output. You can use the Big Data File stage in jobs that run in parallel or sequential mode. However, you cannot use the Big Data File stage in server jobs.

As a target, the Big Data File stage can have a single input link and a single reject link. As a source, you can use the Big Data File stage to write data to one or more files.