Complex Flat File stage
The Complex Flat File (CFF) stage is a file stage. You can use the stage to read a file or write to a file, but you cannot use the same stage to do both.
As a source, the CFF stage can have multiple output links and a single reject
link. You can read data from one or more complex flat files, including MVS data sets with QSAM and VSAM files.
You can also read data from files that contain multiple record types. The source data can contain
one or more of the following clauses:
- GROUP
- REDEFINES
- OCCURS
- OCCURS DEPENDING ON
Note: Although the CFF stage supports OCCURS DEPENDING ON clauses on read, it does not completely
support them on write.
CFF source stages run in parallel mode when they are
used to read multiple files, but you can configure the stage to run sequentially if it is reading
only one file with a single reader.As a target, the CFF stage can have a single input link and a single reject link. You can write data to one or more complex flat files. You cannot write to MVS data sets or to files that contain multiple record types.
CFF stage can recognize hdfs files if the file name begins with hdfs://. For example, if the file name is /tmp/input.txt, you need to specify hdfs:///tmp/input.txt in Complex Flat File stage file name field.
You may need to add the following environment variable for using file pattern on Kerberos environment:
APT_IMPEXP_HDFS_USER=dsadm
