Working with files

You can use the FileInput, FileRead, CDInput, and FTEInput nodes in your message flows to process data from files. Use the FileOutput node, CDOutput, and FTEOutput node to send data from a message flow into a file. Use the FileExists node to poll for the existence of a specified file at a specified location, and either delete the file or leave it in the file directory after it has been propagated through the flow. Use the FileIterator node to propagate lists of files in a message flow, and generate an updated version of the local environment.

About this task

Using files is one of the most common methods of storing data. You can create message flows to process data in files, accepting data in files as input message data, and producing output message data for file-based destinations. The following file nodes are provided:

FileInput node. Use this node to receive messages from files in the file system of the integration node or, by using FTP, FTPS, or SFTP, in a remote file system. The node generates output message data that any of the output nodes can use, which means that messages can be generated for clients using any of the supported transport protocols to connect to the integration node. For more information, see Using a local file as input for your message flow.
FTEInput node. Use this node to start a message flow when files are received over a IBM® MQ File Transfer Edition network. For more information, see Receiving a file by IBM MQ File Transfer Edition.
CDInput node. Use this node to start a message flow when files are received over a IBM Sterling Connect:Direct® network. For more information, see Receiving a file using IBM Sterling Connect:Direct.
FileRead node. Use this node to read data from a file in the middle of a message flow. For more information, see Routing or enriching a message based on the contents of a file.
FileOutput node. Use this node to write messages to a file in the file system of the integration node or, by using FTP, FTPS, or SFTP, in a remote file system. The node can create new files, replace existing files and append data to the end of an existing file. For more information, see Writing a file to your local file system and Writing a file to a remote FTP, FTPS, or SFTP server.
FileExists node. Use this node to poll for the existence of a specified file at a specified location, and either delete the file or leave it in the file directory after it has been propagated through the flow. For more information, see Configuring the FileExists node.
FileIterator node. Use this node to propagate lists of files in a message flow, and generate an updated version of the local environment. For more information, see FileIterator node.
FTEOutput node. Use this node to send a file to a remote destination by using a IBM MQ File Transfer Edition network. For more information, see Sending a file by IBM MQ File Transfer Edition.
CDOutput node. Use this node to send a file to a remote destination by using an IBM Sterling Connect:Direct network. For more information, see Initiating a managed file transfer using IBM Sterling Connect:Direct.

The FileInput and FTEInput nodes start the message flow when a new file arrives, whereas the FileRead node must be connected to another node to start the message flow transaction. The FileRead node also provides keyed access to identify a record, unlike the FileInput node, which processes all records in order. By using these nodes, you can also process large files without the complete message being held in memory, and you can simplify the processing of files that have large numbers of repeating entries.

If you are accessing files on an NFS share, ensure that you are using NFS version 4. The server must support file locking.

work with files, read these topics: