Reading and writing files
Use the stages in the File section of the palette to read and write data from files. These stages are described in detail in the documentation about
connecting to data sources.
Data set stage
The Data Set stage is a file stage that allows you to read data from or write data to a data set.
Sequential file stage
The Sequential File stage is a file stage that allows you to read data from or write data one or more flat files.
File set stage
The File Set stage is a file stage that allows you to read data from or write data to a file set.
Lookup file set stage
The Lookup File Set stage is a file stage that allows you to create a lookup file set or reference one for a lookup. When performing lookups, Lookup File Set stages are used with Lookup stages.
External Source stage
The External Source stage is a file stage that allows you to read data that is output from one or more source programs.
External Target stage
The External Target stage is a file stage that allows you to write data to one or more source programs.
Complex Flat File stage
The Complex Flat File (CFF) stage is a file stage. You can use the stage to read a file or write to a file, but you cannot use the same stage to do both.
Big Data File stage
You can use the Big Data File stage to access files on the Hadoop Distributed File System (HDFS). You use the Big Data File stage to read and write HDFS files.