ac55280_.htm |
Last updated Friday, 21 July 2017The broker reads files with the FileInput, FTEInput, CDInput, and FileRead nodes, and writes files with the FileOutput, CDOutput, and FTEOutput nodes.
The FileInput, FileOutput, and FileRead nodes inherit the permissions that are assigned to the broker's user ID and group. See your operating system documentation for details on how to set permissions and the impact on file access.
You can use the FTEInput and FTEOutput nodes to receive or send files to a destination on a WebSphere® MQ File Transfer Edition network.
You can use the CDInput and CDOutput nodes to receive or send files to a destination on an IBM Sterling Connect:Direct® network.
A file, or a record within a file, is analogous to a message in a queue. The directory that contains the file is analogous to a message queue.
The FileInput node processes messages that are read from files. The FileInput node searches a specified input directory (in the file system that is attached to the broker) for files that match specified criteria. The node can also recursively search the input directory's subdirectories. Files that meet the criteria are processed in age order, that is the oldest files are processed first regardless of where they appear in the directory structure. Optionally, files from a remote FTP or SFTP server can be moved to the local directory whenever the directory is to be scanned. You can find the file that you require by specifying an explicit file name or a file name pattern that includes wildcard characters. If the file is locked, it is ignored during the directory scan. You can also exclude files from being read by specifying a file exclusion pattern. Whether files are either processed or ignored is determined first according to the file name or pattern and second according to the file exclusion pattern.
The FileInput node creates an mqsitransitin subdirectory in the input directory. The mqsitransitin subdirectory holds and locks the input files while they are being processed. The broker reads the file and propagates a message, or messages, by using the contents of the file.
If an integration server that processes files in this input directory is removed, check the mqsitransitin subdirectory for partially processed or unprocessed files. Move any such files back into the input directory and remove the integration server UUID prefix from the file names, so that they can be processed by a different integration server. For more information about the mqsitransitin subdirectory, see How multiple file nodes share access to files in the same directory.
After the file is successfully processed, it is either deleted from the file system or moved to an archive subdirectory of the specified (local) directory.
When the last record of the file is processed successfully, if the End of Data terminal of the FileInput node is connected in the message flow, an End of Data message is propagated to the End of Data terminal. The End of Data message consists of an empty BLOB message and a LocalEnvironment.File structure, and it allows explicit end-of-flow processing to be performed in another part of the message flow.
The message (or messages) propagated from the file can be used as input to any message flow and output node. You can create a message flow that receives messages from files and generates messages for clients that use any of the supported transports to connect to the broker.
Whenever a message is propagated from a file, the FileInput node stores information about the file in the LocalEnvironment.File message tree. This information includes the offset of the record of the message in the file that is being processed, and the record number in that file. In addition, when a wildcard is used in a file name pattern, the characters that are matched in the file name are placed in the WildcardMatch element of the local environment tree.
If a file is backed out, or a file is left in the mqsitransitin subdirectory because processing failed, remove the integration server UUID prefix from the file name and move it back into the input directory. Processing is automatically retried, beginning at the first record in the file.
The FTEInput node receives files from the WebSphere MQ File Transfer Edition network. The broker reads the file and propagates a message, or messages, by using the contents of the file. After an FTEInput node processes a file, the file is deleted. For details of information that the FTEInput node provides in the local environment, see Using local environment variables with file nodes.
The CDInput node receives files from the IBM Sterling Connect:Direct network. The broker reads the file and propagates a message, or messages, by using the contents of the file. After a CDInput node processes a file, the file is deleted. For details of information that the CDInput node provides in the local environment, see Using local environment variables with file nodes.
The FileRead node can read a whole file, or a single, nominated record. A common use of the FileRead node is to route or enrich messages that are based on the contents of the file. When developing the message flow, you can specify the name and location of the file to be read. You can override these values at runtime based on the contents of a message. TheFileRead node reads a file in the middle of a message flow.
The FileRead node complements the existing FileInput and FileOutput nodes.
The FileRead node locks files when they are being read and does not move files to a subdirectory to read them. Files can be read completely or separated into records, in the same way as the FileInput node. After the file is processed by the FileRead node, it can be deleted, archived, or left unchanged.
When a FileRead Node has the Remote Transfer option selected, and a local file is not available, one is transferred from a remote server. Only one thread on a single integration server is permitted to transfer a particular file at any time. Any message flow invocations that require access to this same file wait until an in-flight transfer is completed and then use the local copy of the file. The FileRead node creates a lock in the locks subdirectory of the local input directory to prevent multiple integration servers from transferring the same file at the same time. This locking only functions correctly if the file system correctly implements Posix locking semantics, in particular NFS file systems must be at least NFS v4.
If the Delete remote file after successful transfer property is selected, then after successfully transferring a file it is deleted from the remote server, otherwise it is left in place.
If the FileRead Node has the Finish File Action property set to No Action, then it ensures that the input file referenced by the File Name pattern is unique in the input directory. This behavior means that if more than one file matches in the local directory due to a wildcard match, then an exception is thrown. This behavior is preserved for remote files, so if the Finish File Action is set to No Action and multiple files match the file pattern on the remote server, an exception is thrown. Under these conditions, the broker is unable to determine which of the multiple files present should be processed.
The FileOutput, CDOutput, and FTEOutput nodes write messages to files in the file system of the broker. When a message is received on the In terminal of the node, it creates and writes a file as a series of one or more records. One record is written to a file for every message received. The name of the file is specified either by a file name pattern in the node or an explicit file name that is either specified in the node or derived from the message.
Optionally, the file can be moved to a directory on a remote FTP or SFTPserver. The server is identified by the Remote server and port property on the node. Alternatively, you can override the node property by setting a value in the local environment (see Local environment overrides for the remote server on the FileOutput node).
If the output node produces a single record from the file (whole file), the file is moved immediately to the output directory without requiring a message to be propagated to the Finish File terminal. In this case, any message sent to the Finish File terminal of the node has no effect on any file, but the message is still propagated to a message flow attached to the End of Data terminal.
On the FTP tab, if Retain local file and Stage in the transit directory are both selected, then Action if file exists occurs on the Finish File action.
You can define the action that the node takes with the output file if the final processing of a file fails. The following table describes the outcome for each action taken by the node on final file processing failure given the write mode and the type of transfer:
| Action on final file processing failure | Write Mode | Remote Transfer | Outcome |
|---|---|---|---|
| No Action | Stage in mqsitransit directory | Either | File is left in mqsitransit subdirectory unless Record Definition is Whole File then File is not written to output directory |
| Write directly to the output file | No | File is not written to output directory | |
| Write directly to the output file | Yes | File is left in output directory unless there is a failure writing to the local output directory | |
| Delete | Stage in mqsitransit directory | Either | File is deleted from mqsitransit subdirectory |
| Write directly to the output file | No | File is not written to output directory | |
| Write directly to the output file | Yes | File is deleted from the local output directory | |
| Move To mqsifailure, or move to mqsifailure and add timestampStage in mqsitransit directory | Stage in mqsitransit directory | Either | File is moved from mqsitransit subdirectory to mqsifailure subdirectory |
| Write directly to the output file | No | File is not written to output directory, and mqsifailure subdirectory is not created | |
| Write directly to the output file | Yes | File is moved from the local output directory to mqsifailure subdirectory unless there is a failure that prevents the file being written to the local output directory. In this case, the file can not be moved to mqsifailure as it did not exist in the local output directory. |
The FileOutput node builds a file either in the output directory or in the mqsitransit directory. On the FTP tab, if Action if remote file exists is set to Append, when the Finish File action occurs the file is transferred to the remote machine by using the FTP Append verb. The data is appended to the end of the remote file or, if the file does not exist, the file is created.
Last updated Friday, 21 July 2017