FileOutput node

Use the FileOutput node to write messages to files.

The FileOutput node is available in the following operation modes:

Developer
Application Integration Suite
Standard
Advanced
Express
Scale
Adapter

For more information, see Operation modes.

This topic contains the following sections:

Purpose
Using this node in a message flow
Configuring the FileOutput node
Terminals and properties

Purpose

You can write one or more messages from message flow transactions to a file in the file system of the integration node. Each message, as it is written to a file, is converted to a sequence of bytes called a record. Records are accumulated until a process is triggered that completes the file and places it either in the specified output directory or a remote FTP or SFTP server directory. Properties on the node specify how records are accumulated into files and where the files are placed when they are finished.

The FileOutput node is contained in the File drawer of the palette and is represented in the workbench by the following icon:

FileOutput node icon

Record processing

The FileOutput node writes files as a sequence of one or more records. Each record is generated from a single message received on the In terminal of the node.

By default, each file comprises a single record but properties on the FileOutput node can specify that the file comprises multiple records and how these records are accumulated in a file. If multiple records are written, the FileOutput node starts with an empty file and writes records to it until a message is received by the Finish File terminal. The node does not append new records to a file that exists. Multiple records can be accumulated in a file in the following ways:

Concatenated: The record created from each message is added, unmodified, to the file.
Padded: Each record is adjusted to be a specific length and padded with a padding byte, if necessary, before being added to the file.
Delimited: A delimiter is used to separate or terminate the records as they are added to the file.

For each message received, whether on the In terminal or the Finish File terminal, you can modify the output directory and the name of the file to be written (or finished) by using elements of the message. On the node you can specify these elements, which, by default, identify elements in the local environment, on the Request properties tab.

File processing

The FileOutput node writes accumulated messages to a file, and places it in a specified directory (the output directory) at either of the following times:

After each record, if the file is to contain a single record. (Specify this behavior by setting the Record definition property to Record is Whole File on the Records and Elements tab.)
When the Finish File terminal receives a message.

The name of the output directory and the names of the output files are determined by the node properties that you specify and by elements of the message that is being processed.

The FileOutput node uses subdirectories of the output directory to store files during and after processing. All these subdirectories begin with the prefix mqsi, and include subdirectories called mqsitransit (the transit directory), mqsiarchive (the archive directory), and mqsifailure (the failure directory). Records are not accumulated directly into a file in the output directory but are accumulated in a file in the transit directory. Files are moved from the transit directory to the output directory when the file is complete. If a file that is to be moved to the output directory has the same name as a file that is already there, you can choose whether the file in the output directory is deleted, moved to the archive directory (mqsiarchive), or renamed before being moved to the archive directory.

If a failure occurs during the final transfer of the file to the output directory, you can choose whether the file is deleted, moved to the failure directory (mqsifailure), renamed before being moved to the failure directory, or no action is taken. The chosen action is not effective if the file is being written directly to the output file on your local system and the FTP property is cleared.

You can specify that the FileOutput node transfers files to a remote FTP or SFTP server as part of file processing. If the file is successfully transferred, it can be deleted from the local file system, or, optionally, retained for the rest of the file processing to occur as usual. The server is identified by the Remote server and port property on the node. Alternatively, you can override the node property by setting a value in the local environment. You can also use the local environment to specify commands to run before or after an FTP or SFTP transfer finishes. For more information, see Local environment overrides for the remote server on the FileOutput node.

During the file transfer operation, the FileOutput creates the destination file. However, the destination file is readable before the file transfer is complete. Therefore, ensure that remote applications do not read the file until the file transfer is complete.

When multiple records are written, no file processing occurs until a message is received on the Finish File terminal of the node. Any message received on the Finish File terminal causes the file to be moved from the transit directory to either the specified output directory or to a remote FTP or SFTP directory.

It is not an error if file processing is initiated when there is no file in the transit directory.

If you set the Record definition property to Record is Whole File on the Records and Elements tab, messages received on the Finish File processing are ignored because the file has already been processed.

On Linux®, UNIX, and z/OS® systems, the permissions that new files are created with are determined by the umask value. The minimum permission required by IBM® Integration Bus for all files, including internal configuration files, is for user and group to have read/write access (660). Consequently, the umask value is set to 0006 by default.

You can override the default umask value by using the following environment variables, although IBM Integration Bus will try to ensure that user and group maintain read/write access:

MQSI_SET_DFE_UMASK=nnnn: Using this environment variable, you can override the umask for new files that are created by integration server processes. nnnn is the octal representation of the desired file creation mask.
MQSI_UMASK_COPY=n: If this environment variable is set to any value and MQSI_SET_DFE_UMASK is not set, the umask value is copied from the umask setting of the shell environment, which is often the system default value.

Note:

Care is required when overriding the umask value to ensure that directories and files created by IBM Integration Bus have a minimum permission of 660 and 770 respectively. A good understanding of umask is needed before using the overrides. If the permissions of newly created directories and files is overly restricted, IBM Integration Bus might not be able to restart successfully.

Message propagation

For every message received on the In terminal and successfully processed by the node, a copy is propagated to the Out terminal for further processing if the terminal is attached.

For every message received on the Finish File terminal and successfully processed by the node, a copy is propagated to the End of Data terminal for further processing if the terminal is attached.

When the FileOutput node propagates a message, either to the Out terminal or to the End of Data terminal, it stores information in the LocalEnvironment.WrittenDestination.File message tree. This table describes the LocalEnvironment.WrittenDestination.File elements:

Element Name	Element Data Type	Description
Directory	CHARACTER	Absolute path of the output directory in the form used by the file system of the integration node. For example, on Windows systems, the directory path starts with the drive letter prefix (such as C:).
Name	CHARACTER	Name of the output file.
Action	CHARACTER	Possible values are: Replace if an output file of the same name is replaced. Create if a new output file is created. Append if this message is associated with a record that is added to an output file. Finish if a Finish File message is received and no file is found to finish (for example, if Record is Whole File is specified and a message is sent to the Finish File terminal). Transmit if the file was transferred by FTP or SFTP and the file was not retained.
Timestamp	CHARACTER	The date and time, in character string form, when the node started to process this file. This value prefixes the names of files that are archived if you set the Action if file exists property to Time Stamp, Archive, Replace Existing File and Append to Existing File on the Basic tab.

Multiple instances

Several message flows might write to the same file, which can happen where there are additional instances of the flow, or where multiple flows contain FileOutput nodes. The FileOutput node permits only a single instance, within an integration server and between integration servers, to write to a file at the same time. While a record is being written, all other instances in the integration server must wait. The order in which instances gain access is not defined.

When the file is complete, the first instance to gain access processes it, and other instances do not find the file. The Action element of the LocalEnvironment.WrittenDestination.File message tree is set to Finish for all instances that fail to discover the file in the transit directory.

Using this node in a message flow

The FileOutput node can be used in any message flow that sends messages to files. See Working with files.

If you configure a File node to use FTP, your network might need it to connect to an FTP proxy server, instead of directly to the remote FTP server. How you configure the File nodes to use an FTP proxy depends on how that proxy handles requests. For some FTP proxies, you must encode the target FTP server information in the logon credentials that you create with the mqsisetdbparms command. For example, some FTP proxies support the following values:

Username: FtpTargetHostUsername@ProxyUserName@TargetFtpHostname 
Password: TargetFtpUserPassword@ProxyUserPassword

Other proxies might require different encodings, or might require external configuration, or you might not be able to use them with the File nodes.

Configuring the FileOutput node

When you have put an instance of the FileOutput node into a message flow, you must configure it (for more information, see Configuring a message flow node). The properties of the node are displayed in the Properties view. All mandatory properties for which you must enter a value (those properties that do not have a default value defined) are marked with an asterisk in that view.

Terminals and properties

The FileOutput node terminals are described in the following table.

Terminal	Description
In	The input terminal that accepts a message for processing by the node.
Finish File	The input terminal that accepts a message that triggers the final processing of a file.
Out	The message received on the In terminal is propagated to this terminal if the record is written successfully. The message is unchanged except for status information in the Local Environment.
End of Data	The message received on the Finish File terminal is propagated to this terminal if the file is processed successfully.
Failure	The output terminal to which the message is routed if a failure is detected when a message is propagated.

The following tables describe the node properties that you can set on a specified tab. The column headed M indicates whether the property is mandatory (marked with an asterisk if you must enter a value when no default is defined); the column headed C indicates whether the property is configurable (you can change the value when you add the message flow to the BAR file to deploy it).

The FileOutput node Description properties are described in the following table.

Property	M	C	Default	Description
Node name	No	No	FileOutput	The name of the node.
Short Description	No	No		A brief description of the node.
Long Description	No	No		Text that describes the purpose of the node in the message flow.

The FileOutput node Basic properties are described in the following table.

Property	M	C	Default	Description	mqsiapplybaroverride command property
Action on final file processing failure	No	No	No Action	The action that the node takes with the output file if the final processing of a file fails. Valid options are: No action Move to mqsifailure Subdirectory The file is moved to the failure subdirectory of the output directory. The name of this subdirectory is mqsifailure. If the input directory is /var/fileinput, the absolute path of the failure subdirectory is /var/fileinput/mqsifailure. If this subdirectory does not exist, the broker creates it when it first tries to move a file there. If the file cannot be moved to this subdirectory, perhaps because a file of the same name exists there, the node adds the current date and time to the file name and makes a second attempt to move the file. If this second attempt fails, the node stops processing. Messages BIP* and BIP** are issued. Resolve the problem with the subdirectory or file before attempting to restart the message flow. Delete The file is deleted if it can not be written to the output directory following a message received on the Finish File terminal. Add Time Stamp and Move to mqsifailure Subdirectory the current date and time are added to the file name, and then the file is moved to the failure subdirectory.
Directory	No	Yes	None	Specify the output directory in which the FileOutput node places its files. Specify the directory as an absolute or relative directory path. If the directory path is relative, it is based on the directory specified in the environment variable MQSI_FILENODES_ROOT_DIRECTORY. For example: On Windows: C:\fileoutput On UNIX: /var/fileoutput To write files in the directory that is identified by MQSI_FILENODES_ROOT_DIRECTORY, ensure that you specify a value of . (a period) in this property. You can override the output directory path to be used by setting values in the current message. For more information, see the Request tab properties.	outputDirectory
File name or pattern	No	Yes	None	Specify a file name pattern. This property defines the name of the file that is created by the FileOutput node. The value is either a specific file name or a character sequence (pattern) that matches a file name. Only patterns with a single wildcard character (the asterisk, *) are allowed in this property field. The file name to be used is determined in the following way: If the file name contains no wildcard, the value of this property is the name of the file created. This value must be a valid file name on the file system or the FTP file system that hosts the integration node to which the message flow is deployed. If the file name contains a single wildcard character, the value of the element LocalEnvironment.Wildcard.WildcardMatch in the current message replaces the wildcard character, and the resulting value is the name of the file created. This value must be a valid file name on the file system or the FTP file system that hosts the integration node to which the message flow is deployed. If the WildcardMatch value is not found, the wildcard character is replaced by an empty string. You can override the name of the file by setting values in the current message. For more information, see the Request tab properties. If the File name or pattern property is empty, you must override the name by using the current message. Wildcard substitution occurs only if this property is not overridden in this way. File names are passed to the file system to which the integration node has access and must adhere to the conventions of these file systems. For example, file names on Windows systems are not case-sensitive; while on UNIX systems, they are.	outputFilename
Mode for writing to file	Yes	No	Stage in transit directory	Specify if the file must be staged or written to directly. Select one of the following options: Stage in transit and move to output directory on Finish File Write directly to the output file
Action if file exists	Yes	No	Replace Existing File	Specify how the file is to be processed when it is complete. Select one of the following options: Replace Existing File (the default value) specifies that if a file of the same name exists in the output directory, the new file replaces it. Append to Existing File moves the output file to the transit directory and appends the file contents to the file. The file is moved back to the output directory when the append is finished. Fail if File Exists specifies that a new file is created, and that if a file of the same name exists in the output directory, the new file remains in the transit directory and exception BIP3307 is produced. Archive and Replace Existing File specifies that if any file of the same name exists in the output directory, it is moved to the archive directory before the new file is placed in the output directory. If any file of the same name exists in the archive directory, an exception is produced. Time Stamp, Archive, and Replace Existing File specifies that if a file of the same name exists in the output directory, its name is augmented with a time stamp (a character-based version of the date and time) before being moved to the archive directory. The format of the time stamp is `yyyyMMdd_HHmmss_SSSUUU`, in UTC time, where `UUU` is an additional ID to ensure that the time stamp is unique.
Replace duplicate archive files	Yes	No	Cleared	Select the Replace duplicate archive files check box to specify that, in cases where Archive and Replace Existing File or Time Stamp, Archive and Replace Existing File is specified in Action if file exists, files moved to the archive directory replace files that exist there already with the same name. By default, this check box is cleared. If this check box is not selected, and there is already a file in the archive directory with the same name as a file that is to be moved there, an exception is produced, and the new file remains in the transit directory.

The FileOutput node Request properties are described in the following table.

These properties specify the location of the data to be written, and control information that overrides the Directory and File name or pattern properties on the Basic tab. You can specify the properties on this tab as XPath or ESQL expressions. Content assist is available in the properties pane and also in the XPath Expression Builder, which you can open by using the Edit button to the right of each property.

Property	M	C	Default	Description	mqsiapplybaroverride command property
Data location	Yes	No	$Body	Specify the input data location, which is the location in the input message tree that contains the record to be written to the output file. The default value is $Body, meaning the entire message body ($InputRoot.Body). When you specify this property, and the data in the message tree that it identifies is owned by a model-driven parser, such as the MRM parser or XMLNSC parser, consider the following factors. If you are using MRM CWF format, ensure that the identified message tree exists as a message definition. If this element is defined as a global element only, exceptions BIP5180 and BIP5167 are produced. If you are using MRM TDS format, serialization of the identified message is successful if the element is defined as a global element or message. However, if the identified field is not found as a global element or message, the following conditions apply: If the field is a leaf field in the message tree, the field is written as self-defining. No validation occurs even if validation is enabled. If the field is a complex element, an internal exception is generated (BIP5522), indicating that the logical type cannot be converted to a string. If you are using MRM XML, the events are similar as for the MRM TDS format except that, if the field is a complex element, it is written as self-defining. If you use the XMLNSC parser, no validation occurs even if validation is enabled.
Request directory property location	Yes	Yes	$LocalEnvironment/Destination/File/Directory	Specify the location of the value to override the Directory property on the Basic tab. If you do not specify a location, the default value is $LocalEnvironment/Destination/File/Directory. If you specify a location but the element is empty or missing, the Directory property is used. The element has a data type of CHARACTER and is an absolute or relative directory path. Use the path separator character ('/' or '\') according to the file system on which the integration node runs. Trailing path separator characters are ignored. Relative directory paths are based on the value of the MQSI_FILENODES_ROOT_DIRECTORY environment variable.	requestDirectoryLocation
Request file name property location	Yes	Yes	$LocalEnvironment/Destination/File/Name	Specify the location of the value to override the File name or pattern property on the Basic tab. The element has a data type of CHARACTER and is an explicit file name. No wildcard substitution occurs for this value. If the property is not in the local environment when the message arrives in the In or Finish File terminal of the node, then the default is to use the File name or pattern property on the Basic tab.	requestNameLocation

The FileOutput node Records and Elements properties are described in the following table.

These properties specify how the FileOutput node writes the record derived from the message.

Property	M	C	Default	Description
Record definition	Yes	No	Record is Whole File	Specify how the records are placed in the output file. Select one of the following options: Record is Whole File specifies that the file is to contain a single record. The file is finished immediately after the record is written; the FileOutput node does not wait for a message on the Finish File terminal. This value is the default. Record is Unmodified Data specifies that records are accumulated in a file with neither padding or delimiters applied. The file is finished only when a message is received on the Finish File terminal. Record is Fixed Length Data specifies that records are padded to a given length if necessary and accumulated in a file by concatenation. You specify this length in the Length property. If the record is longer than the value specified in Length, the node produces an exception. Use the Padding byte property to specify the byte to be used for padding the message to the required length. Records are added to this file until a message is received on the Finish File terminal. Record is Delimited Data to specify that records are separated by a delimiter and accumulated by concatenation. The delimiter is specified by the Delimiter, Custom delimiter, and Delimiter type properties. Records are added to this file until a message is received on the Finish File terminal.
Length	Yes	No	80	Specify the length (in bytes) of records when Record is Fixed Length Data is specified in Record definition. Records that are longer than this value cause an exception to be produced. This value must be in the range 1 byte through 104857600 bytes (100 MB). The default value is 80 bytes.
Padding byte	Yes	No	X'20'	When Record is Fixed Length Data is specified in Record definition, use the Padding byte property to specify the byte to be used when padding records to the specified length if they are shorter than this length. Specify this value as two hexadecimal digits. The default value is X'20'.
Delimiter	Yes	No	Broker System Line End	Specify the delimiter to be used if you specify Record is Delimited Data in Record definition. Select one of the following options: Broker System Line End specifies that a line end sequence of bytes is used as the delimiter, as appropriate for the file system on which the integration node is to run. For example, on Windows systems, the delimiter is a 'carriage-return, line-feed' pair (X'0D0A'); on UNIX systems, it is a single 'line-feed' byte (X'0A'); on z/OS systems, it is a 'newline' byte (X'15'). This value is the default. Custom Delimiter specifies that the explicit delimiter sequence defined in the Custom delimiter property is to be used to delimit records.
Custom delimiter	No	No	None	Specify the delimiter sequence of bytes to be used to delimit records when Custom Delimiter is specified in the Delimiter property. Specify this value as an even-numbered string of hexadecimal digits. The default value is X'0A' and the maximum length of the string is 16 bytes.
Delimiter type	Yes	No	Postfix	If you set the Record definition property to Record is Delimited Data, use Delimiter type to specify how the delimiter is to separate records. Select one of the following options: Postfix specifies that the delimiter is added after each record that is written. This value is the default. Infix specifies that the delimiter is inserted only between any two adjacent records.

The FileOutput node Validation properties are described in the following table.

For a full description of these properties, see Validation properties.

Property	M	C	Default	Description	mqsiapplybaroverride command property
Validate	No	Yes	Inherit	Specify whether validation takes place. Valid values are: None Content and Value Content Inherit	validateMaster
Failure action	No	No	Exception	Specifies what happens if validation fails. You can set this property only if you set Validate to Content or Content and Value. Valid values are: User Trace Local Error Log Exception Exception List

The FileOutput node FTP properties are described in the following table.

To transfer files to an FTP, FTPS, or SFTP server, select the Remote Transfer property, then set the properties described in this table.

Property	M	C	Default	Description	mqsiapplybaroverride command property
Remote transfer	No	Yes	Cleared	To transfer files to an FTP, FTPS, or SFTP server, select Remote Transfer, then set the other properties in this table.	fileFtp
Transfer protocol	No	Yes	FTP	This property specifies the protocol to be used for remote transfer. Valid values are: FTP SFTP FTPS	remoteTransferType
Remote server and port	No	Yes	None	This property can have either of the following values: The IP address or name (and, optionally, the port number) of a remote FTP, FTPS, or SFTP server The name of a configurable service of type FtpServer Specify the IP address and port number of the FTP, FTPS, or SFTP server to be used, by using the following syntax: `IP_address_or_URL` or `IP_address_or_URL:port_number` If you specify the IP address in IPv6 format, ensure that you enclose it in square brackets, for example: `[12a::13bd:24cd]` or `[12a::13bd:24cd]:123` where `123` is the port number If you are using FTP or FTPS and you do not specify a port number, 21 is assumed. If you are using SFTP and you do not specify a port number, a port number of 22 is assumed. However, if an FtpServer configurable service is defined, you can enter the name of the configurable service in this field. If a configurable service name is specified, any or all of the other remote transfer properties on the FTP tab can be overridden by the configurable service. For information about how an FtpServer configurable service definition and the properties on this tab interact, see FtpServer configurable service. You can override this property by setting the location of the server in the local environment. For more details, see Local environment overrides for the remote server on the FileOutput node.	fileFtpServer
Security identity	No	Yes	None	Specify the name of a security identity that has been defined by using the mqsisetdbparms command. The user identifier and password that are to be used to log on to the FTP, FTPS, or SFTP server are obtained from this definition. The name of the definition must have the prefix `ftp::`. The value of this property is overridden by the value in the FtpServer configurable service property securityIdentity, if it is set.	fileFtpUser
Server directory	No	Yes	"."	Specify the directory on the FTP, FTPS, or SFTP server to which to transfer files. The default value is `.` (a period), which indicates the default directory after logon. If you specify a relative path, the directory is based on the default directory after FTP, FTPS, or SFTP logon. Ensure that the syntax of the path conforms to the file system standards in the FTP, FTPS, or SFTP server. The value of this property is overridden by the value in the remoteDirectory property of the FtpServer configurable service, if it is set. If the directory specified in this field does not exist on the remote server, the node attempts to create it.	fileFtpDirectory
Transfer mode	No	Yes	Binary	Specify how files are transferred. If the file contents are not transformed, select Binary. If the file is transmitted as ASCII, select ASCII. The value of this property is overridden by the value in the FtpServer configurable service property transferMode, if it is set. This property is valid only when FTP is selected as the protocol for remote transfer. If you have specified FTPS or SFTP as the protocol, the Transfer mode property is ignored and Binary encoding is used.
Action if remote file exists	No	No	Replace File	Specify whether to create the file or append to an existing file. Select one of the following options: Replace Existing File (the default value) specifies that if a file of the same name exists in the output directory, the new file replaces it. The file is replaced by using the FTP put verb. Append to Existing File moves the output file to the transit directory and appends the file contents to the file. The file is transferred to the remote machine by using the FTP append verb and moved back to the output directory when the append is finished.
Retain local file after transfer	No	No	Cleared	To retain a local copy of the file after the file transfer process has completed, select the Retain local file after transfer check box. If this check box is selected, the local copies are processed after the transfer is complete, as are other output files, as specified on the Basic tab. If the check box is cleared, successfully transferred files are not retained locally.

The Monitoring properties of the node are described in the following table.

Property	M	C	Default	Description
Events	No	No	None	Events that you have defined for the node are displayed on this tab. By default, no monitoring events are defined on any node in a message flow. Use Add, Edit, and Delete to create, change or delete monitoring events for the node; see Configuring monitoring event sources by using monitoring properties for details. You can enable and disable events that are shown here by selecting or clearing the Enabled check box.