FileRead node

Use the FileRead node to read one record, or the entire contents of a file, from within a message flow.

The FileRead node is available in the following operation modes:
  • Developer
  • Application Integration Suite
  • Standard
  • Advanced
For more information, see Operation modes.

This topic contains the following sections:

Purpose

You can use the FileRead node to read one record, or the entire contents of a file, from within the middle of a message flow, see Routing or enriching a message based on the contents of a file. The FileRead node is like the FileInput node, which reads a file from the start of a message flow, except it is driven to read the file by an incoming message. You can configure the FileRead node to propagate a whole file or part of a file, and the FileRead can also transfer a file from a remote FTP, FTPS, or SFTP server if a local file is not available.

The FileRead node is contained in the File drawer of the palette, and is represented in the IBM® Integration Toolkit by the following icon:

FileRead node icon

Using this node in a message flow

To enable function that becomes available in IBM Integration Bus fix packs, use the -f parameter on the mqsichangebroker command. For more information, see mqsichangebroker command.

By using any built-in parser, the FileRead node can parse the contents of the file and propagate the contents as a message tree. The node streams data to parsers which support this function, in an identical way to the FileInput node.

The main properties of the node specify the file and directory to read the file from. The file name is given as a pattern which can include wildcards. Both the directory and file pattern can be overridden by using fields in the local environment.

Terminals and properties

The FileRead node input terminals are described in the following table.

Terminal Description
In The input terminal that accepts a message for processing by the node.
Finish file in The input terminal that accepts a request to perform the finish file action, without reading any data.

When a message is received by the Finish file in terminal, the FileRead node starts the processing that is specified by the Action property. When a message is received by the Finish file in terminal, no data is read before the action is taken.

The FileRead node output terminals are described in the following table.

Terminal Description
Failure The output terminal to which a message is routed if a failure is detected when the message is propagated.
Out The output terminal to which a message is routed if it is successfully retrieved from an external resource. If no errors occur in the input node, a message received from an external resource is always sent to the Out terminal first.
No match The message received on the No match terminal is propagated to this terminal if the file does not exist on the file system or it does exist but no record which matches the filter expression can be found.

If the terminal is not connected, the message is not used.

Finish file out A message arriving on the Finish file in terminal is propagated to the Finish file out terminal with the content unchanged, but the local environment is updated with details of the action the node has taken.

The following tables describe the node properties that you can set on a specified tab. The column headed M indicates whether the property is mandatory (marked in the IBM Integration Toolkit with an asterisk if you must enter a value when no default is defined). The column headed C indicates whether the property is configurable (you can change the value when you add the message flow to the BAR file to deploy it).

When the contents of the file have been successfully propagated down the flow, the file is deleted from the file system. No other file node can access the file when the read node starts reading data from it. If a file does not exist which matches the pattern, then the original message is propagated to the 'No match' terminal. If the terminal is not connected, the message is not used.

At the end of processing, it is possible to configure the node not to delete the file. In this mode, any other file read node can also access the file if running in the same browse only mode.

The Description properties of the FileRead are described in the following table.

Property M C Default Description
Node name No No FileRead The name of the node.
Long Description No No None Text that describes the purpose of the node in the message flow.
Short Description No No None A brief description of the node.

The Basic properties of the FileRead are described in the following table.

Property M C Default Description mqsiapplybaroverride command property
Input directory No Yes None Absolute path of the input directory in the form used by the file system of the integration node. For example, on Windows systems, the directory path starts with the drive letter prefix (such as C:).

Alternatively, the path can be relative to the file nodes root directory (which can be overridden with the same environment variable as used for the File input and output nodes).

inputDirectory
File name or pattern No Yes * A file name, or a character sequence (pattern) that matches a file name. A pattern contains at least one of the following wildcard characters:
  • Asterisk (*), representing any sequence of zero or more characters
  • Question mark (?), representing any single character

If more than one file matches the pattern, and the Action property is set to No action - do nothing to the file, an exception is thrown.

Select the Use environment wildcard option if you want to use part of the file name from the input node in this field.

filenamePattern
Substitute wildcard match No No False If you select this option, the file pattern is no longer a regular expression. Instead, it must contain only one *. The * is replaced by the $LocalEnvironment/Wildcard/WildcardMatch field. This location is normally populated by a previous node, such as the FileInput node. substituteWildcardMatch
Action Yes No No action The action performed to the file after either the end of the file is reached, or a message is received by the Finish file in terminal. This action occurs in any of the following circumstances:
  1. When a file is read, if Record detection is set to Whole File.
  2. When the record read is the last record in the file, if Record detection is set to anything other than Whole File.
  3. When a message is received by Finish file in. In this case no data is read before the action is performed.
Valid actions are:
  • No action - do nothing to the file.
  • Delete - delete the file.
  • Archive - move the file to an archive directory.
  • Archive with timestamp - move the file to the archive directory and add a timestamp.

By default if move to Archive is selected the file is moved to the mqsiarchive sub directory, but this default can be overridden by setting a local environment variable.

When the local environment override for the archive file name is set and the finish file action on the node is set to Archive with timestamp, the file name is timestamp_nameYouSpecified. Where timestamp is the date and time the file is archived and nameYouSpecified is the name you give to the file.

When the local environment override for the archive name is set and the finish file action on the node is set to Delete, the file is deleted.

When the local environment override for the archive name is set and the finish file action on the node is set to No action, no action is applied to the file.

 
Replace duplicate archives Yes No False By default, an error occurs if a file is archived and a file exists with the same name. Set this property to true to ignore the error and replace the archive file.  

The Request properties of the FileRead are described in the following table.

Property M C Default Description mqsiapplybaroverride command property
Request directory property location No No $LocalEnvironment/Destination/File/Directory The message element location containing the name of the input directory.  
Request file name property location No No $LocalEnvironment/Destination/File/Name The message element location containing the name of the input file pattern.  
Offset property location No No $LocalEnvironment/Destination/File/Offset The message element location containing the offset to start searching for records from.  
Length property location No No $LocalEnvironment/Destination/File/Length The message element location containing the length of the record to read if using fixed-length record detection.  

The Result properties of the FileRead are described in the following table.

Property M C Default Description mqsiapplybaroverride command property
Result data location Yes No $ResultRoot The part of the record that is copied to the location in the outgoing message that is specified by the "Output data location" property. The default value specifies that the entire record is copied.

See Combining a result message with an input message when fetching data from external systems.

 
Output data location Yes No $OutputRoot The location in the outgoing message where the record that is retrieved from the file is copied. The part of the record that is copied is specified by the "Result data location" property. The default value specifies that the retrieved record is copied to the root of the outgoing message.

See Combining a result message with an input message when fetching data from external systems.

 
Copy local environment No No Selected Specifies whether the local environment is copied to the output message.
  • If Copy local environment is selected, a new copy of the local environment is created in the tree, and it is populated with the contents of the local environment from the preceding node. Therefore, if a node changes the local environment, the upstream nodes are not affected by those changes because they have their own copies. This value is the default.
  • If Copy local environment is cleared, the node does not generate its own copy of the local environment, but uses the local environment that is passed to it by the preceding node. Therefore, if a node changes the local environment, the changes are reflected by the upstream nodes.
 
Record selection expression No No true() The expression that is used to select the correct record from the file. The expression is evaluated for each record in the file until one is found that evaluates to true. That record is then propagated to the Out terminal.

The expression can be set to any valid XPath expression that returns a Boolean value.

The expression is not used when Whole File is selected as the Record Detection option.

The following correlation names are in scope to use in the expression:
  • InputRoot
  • InputLocalEnvironment
  • OutputRoot
  • OutputLocalEnvironment
  • ResultRoot
 

The Input Message Parsing properties of the FileRead are described in the following table.

Property M C Default Description mqsiapplybaroverride command property
Message domain No No   The domain that is used to parse the incoming message.  
Message model No No   The name or location of the message model schema file in which the incoming message is defined.

If you set this property, and then update the project dependencies to remove this message model reference, a warning is issued. Either update the Message model property, or restore the reference to this message model.

 
Message No No   The name of the incoming message.  
Physical format No No   The name of the physical format of the incoming message.  
Message coded character set ID No Yes Broker System Default The ID of the coded character set used to interpret bytes of the file being read. messageCodedCharSetIdProperty
Message encoding No Yes Broker System Determined The encoding scheme for numbers and large characters used to interpret bytes of the file being read. Valid values are Broker System Determined or a numeric encoding value. For more information about encoding, see Data conversion. messageEncodingProperty

The Parser Options properties of the FileRead are described in the following table.

Property M C Default Description
Parse timing No No On Demand This property controls when an input message is parsed. Valid values are:
  • On Demand
  • Immediate
  • Complete

For a full description of this property, see Parsing on demand.

Build tree using XML schema data types No No Cleared This property controls whether the syntax elements in the message tree have data types taken from the XML schema.
Use XMLNSC compact parser for XMLNS domain No No Cleared This property controls whether the XMLNSC Compact Parser is used for messages in the XMLNS Domain. If you set this property, the message data is shown under XMLNSC in nodes that are connected to the output terminal when the input MQRFH2 header or Input Message Parsing property, Message Domain is XMLNS.
Retain mixed content No No Cleared This property controls whether the XMLNSC parser creates elements in the message tree when it encounters mixed text in an input message. If you select the check box, elements are created for mixed text. If you clear the check box, mixed text is ignored and no elements are created.
Retain comments No No Cleared This property controls whether the XMLNSC parser creates elements in the message tree when it encounters comments in an input message. If you select the check box, elements are created for comments. If you clear the check box, comments are ignored and no elements are created.
Retain processing instructions No No Cleared This property controls whether the XMLNSC parser creates elements in the message tree when it encounters processing instructions in an input message. If you select the check box, elements are created for processing instructions. If you clear the check box, processing instructions are ignored and no elements are created.
Opaque elements No No Blank This property is used to specify a list of elements in the input message that are to be opaquely parsed by the XMLNSC parser. Opaque parsing is performed only if validation is not enabled (that is, if Validate is None); entries that are specified in Opaque Elements are ignored if validation is enabled.

The Records and Elements properties of the FileRead are described in the following table.

Property M C Default Description
Record detection Yes No Whole File The mechanism used to identify records in the input file. Valid options are:
  • Whole File
  • Fixed Length
  • Delimited
  • Parsed Record Sequence
Note: If you select the Parsed Record Sequence option, the parser specified in Message domain must be DFDL, XMLNSC or MRM.
Length Yes No 80 The length of each record, in bytes, when Fixed Length record detection is selected.
Delimiter Yes No DOS or UNIX Line End The type of delimiter bytes that separate, or end, each record when Delimited record detection is selected. Valid options are:
  • DOS or UNIX Line End
  • Custom Delimiter
You can set the delimiter only when the Record detection property is set to Delimited.
Custom delimiter No Yes   The delimiter bytes, expressed in hexadecimal, when Delimited record detection and Custom Delimiter are selected. This property is mandatory only if the Delimiter property is set to Custom Delimiter.
Delimiter type Yes No Postfix The position of the delimiter when Delimited record detection is selected. Valid options are:
  • Postfix
  • Infix
This property is ignored unless the Delimiter property is set to Custom Delimiter.

The Validation properties of the FileRead node are described in the following table. For a full description of these properties, see Validation properties

Property M C Default Description mqsiapplybaroverride command property
Validate No Yes None This property controls whether validation takes place. Valid values are:
  • None
  • Content and Value
  • Content
validateMaster
Failure action No No Exception This property controls what happens if validation fails. Valid values are:
  • User Trace
  • Local Error Log
  • Exception
  • Exception List
 

The FTP properties of the FileRead node are described in the following table.

Property M C Default Description mqsiapplybaroverride command property
Remote Transfer No Yes Cleared This property defines whether the node uses the remote file transfer properties that are listed on the FTP tab, and if the node reads files from either an FTP, FTPS, or SFTP server.

If you selected this property and the Include local subdirectories property, only the top-level directory that you specified on the remote system is searched for files.

fileFtp
Transfer protocol No Yes FTP This property specifies the protocol to use for remote transfer. Valid values are:
  • FTP
  • FTPS
  • SFTP
remoteTransferType
Remote server and port No Yes None This property can have either of the following values:
  • The IP address or name of a remote FTP, FTPS, or SFTP server, and an optional port number. For example: ftp.server.com:21 or sftp.server.com:22
  • The name of a configurable service of type FtpServer
If a configurable service name is specified, any or all the other remote transfer properties on the FTP tab can be overridden by the configurable service.

You can override this property by setting the location of the server in the local environment. For more information, see Local environment overrides for the remote server on the FileOutput node.

fileFtpServer
Security identity No Yes The name of the user ID that was used to access the FTP or SFTP server. This property is overridden by the securityIdentity property in the FtpServer configurable service, if it is set. fileFtpUser
Server directory No Yes "." The directory on the FTP, FTPS, or SFTP server that files are transferred from. If you specify this property as a relative path, it is relative to the home directory after logon. This property is overridden by the remoteDirectory property in the FtpServer configurable service, if it is set. If the directory specified in this field does not exist on the remote server, the node attempts to create it. fileFtpDirectory
Transfer mode No No Binary The FTP transfer mode for transfer of file data. This property is valid only when FTP is selected as the protocol for remote transfer. Valid values are:
  • Binary
  • ASCII
This property is overridden by the transferMode property in the FtpServer configurable service, if it is set..

If you have specified SFTP as the protocol for remote transfer, the Transfer mode property is ignored, and binary encoding is used instead.

 
Delete remote file after successful transfer No Yes No This property control what happens to the remote file on the FTP or SFTP server after a successful file transfer. Valid values are:
  • Yes If set to Yes, the file is deleted from the remote server.
  • No If set to No, the file is left on the remote server
deleteRemoteFile