Configuring the FileExists node

When you add a FileExists node to a message flow, you can configure it to poll for the existence of a specified file or a list of files at a specified location, and to delete or leave the them in the file directory without processing the file contents.

1. Optional: On the Description tab, enter a Short description, a Long description, or both.
   You can also rename the node on this tab.
2. On the Basic tab, enter the directories and files to be processed by the FileExists node, together with what to do with any duplicate files encountered. You can also specify whether the files are to be deleted or left in the file directory after they are processed by the flow.
   1. In the Input directory field, specify the directory from which the FileExists node obtains files. Specify the directory as either an absolute or a relative directory path. If the directory path is relative, it is based on the directory that is specified in the environment variable MQSI_FILENODES_ROOT_DIRECTORY. An example on Windows systems is C:\fileinput. An example on UNIX systems is /var/fileinput.

      On Windows, if you are specifying a shared directory that is mapped to your local computer, specify the share name instead of the letter that represents the drive; for example, \\myshare\mydirectory.

   2. Select Include local subdirectories if you want files in the directory structure under the Input directory to be processed. The integration server must have access to the top-level directory that you specify. If the integration server does not have access to a particular subdirectory, that subdirectory is not searched, and a message is written to user trace. Files are processed according to their age. The oldest files are processed first regardless of where they appear in the directory structure. The decision about whether files are processed or ignored is determined firstly according to the file name or pattern and secondly according to the file exclusion pattern.
   3. Select List mode if you want the FileExists node to scan all the files in the input directory and propagate a list of files, rather than propagating information about the next file in the directory. By default, this option is not selected, which means that the FileExists node propagates information about the next file in the input directory, rather than a list.

      The other properties set on the node (such as filename patterns and subdirectory settings) apply regardless of whether the List mode property is set.

      The FileIterator node can be used to iterate over one or more files and produce a LocalEnvironment that is compatible with the FileRead node.

      The LocalEnvironment for one file (not using list mode), has the following form:

      
      (0x01000000:Name):File     = (
          (0x03000000:NameValue):Directory    = 'C:\Work\Development\V12\FileNodes\Exists' (CHARACTER)
          (0x03000000:NameValue):Name         = 'InputMsg.xml' (CHARACTER)
          (0x03000000:NameValue):LastModified = GMTTIMESTAMP '2022-01-26 11:26:32.025' (GMTTIMESTAMP)
          (0x03000000:NameValue):IsEmpty      = FALSE (BOOLEAN)
          (0x03000000:NameValue):TimeStamp    = '20220327_092919_509715' (CHARACTER)
          (0x03000000:NameValue):ProcessedBy  = 'File Exists' (CHARACTER)
          (0x03000000:NameValue):FromFTP     = FALSE (BOOLEAN)
        )
        (0x01000000:Name):Wildcard = (
          (0x03000000:NameValue):WildcardMatch = 'InputMsg.xml' (CHARACTER)
        )

      The LocalEnvironment for a list of files has the following form:

      
      (0x01000000:Name):Files     = (
        (0x03000000:NameValue):TimeStamp    = '20220327_092919_509715' (CHARACTER)
        (0x03000000:NameValue):ProcessedBy  = 'File Exists' (CHARACTER)
        (0x03000000:NameValue):FromFTP     = FALSE (BOOLEAN)
        (0x03000000:NameValue):TotalInList = 1 (INTEGER)
        (0x01000000:Name):File     = (
          (0x03000000:NameValue):Directory    = 'C:\Work\Development\V12\FileNodes\Exists' (CHARACTER)
          (0x03000000:NameValue):Name         = 'InputMsg.xml' (CHARACTER)
          (0x03000000:NameValue):LastModified = GMTTIMESTAMP '2022-01-26 11:26:32.025' (GMTTIMESTAMP)
          (0x03000000:NameValue):IsEmpty      = FALSE (BOOLEAN)
          (0x01000000:Name):Wildcard = (
            (0x03000000:NameValue):WildcardMatch = 'InputMsg.xml' (CHARACTER)
          )
        )
      )

   4. In File name or pattern, specify a pattern for the file name. It is either a file name or a character sequence (a pattern) that matches a file name. A pattern is a sequence that contains at least one of the following wildcard characters:
   5. In File exclusion pattern, specify a pattern for file names that you want to exclude from processing. Specify either a file name or a character sequence (a pattern) that matches a file name. For a file to be excluded from processing, its name must match the pattern.
   6. The Delete file property controls whether the file is deleted when it has been propagated through the FileExists node. By default, this property is set to Always, which means that the list of files is removed from the input directory or server directory after it has been propagated through the node, before the directory is scanned again.

      If you want to leave the file in the directory after it has been propagated through the FileExists node, set the Delete file property to Never. If you set this property to Never, the message flow implementation is responsible for deleting, renaming, or moving the files, which you can do by using a FileRead node or a JavaCompute node, for example.

      When the list of files has been propagated to the available terminals, the FileExists node checks whether all the current files in the list have been removed from the input directory. If the node detects the same files again, and if they are unchanged from when they were previously listed, the FileExists node blocks further processing. This blocking continues until the files are manually deleted, removed, or renamed. To prevent this blocking, ensure that the Delete file property is set to Never only if other nodes in the message flow process the file, so that it does not remain in the input or server directory.

      For more information, see the description of the properties in FileExists node.
   7. If user tracing is in operation, the FileExists node writes a message to the user trace whenever it processes a file or a list of files.
   8. If activity logging is in operation, the FileExists node writes a message to the activity log whenever it processes a file or a list of files.
3. On the Polling tab, enter a value for the Polling interval. This property controls the frequency with which the FileExists node accesses the file system to search for files to process.

   After the initial scan of the directory when the flow is started, whenever the directory is found to contain no files that match the input pattern, the FileExists node waits for the period that is defined by this property. This process avoids the need for the FileExists node to be continually accessing the file system, and consuming large amounts of system resource.

   The smaller the value set in this property, the more quickly the FileExists node discovers files that are in the input directory. However, a smaller value increases the use of system resources. A larger value reduces the use of system resource but at the cost of the FileExists node discovering files to process less quickly.

   Do not use this property as a means to regulate work, or to schedule processing. If you want the FileExists node to monitor the input directory for selected periods only, start and stop the message flow at appropriate times.

   If you select the Remote Transfer property and set the Scan delay property on the FTP tab, the value that you set for Scan delay overrides the value set for Polling interval.

4. Use the Retry tab to define how retry processing is performed when a message flow fails.
   1. Retry mechanism determines the action that occurs if the flow fails:
      • Select Failure for the node to report a failure without any retry attempts.
      • Select Short retry for the node to try again before reporting a failure if the condition persists. The number of times that it tries again is specified in Retry threshold.
      • Select Short and long retry for the node to try again, first using the value in Retry threshold as the number of attempts it is to make. If the condition persists after the Retry threshold value has been reached, the node then uses the value of Long retry interval between attempts.
   2. Specify a value for the Retry threshold property. The number of times the node tries the flow transaction again if the Retry mechanism property is set to either Short retry or Short retry and long retry.
   3. Specify a value for the Short retry interval property. The length of time, in seconds, to wait between short retry attempts.
   4. Specify a value for the Long retry interval property. The length of time to wait between long retry attempts until a message is successful, the message flow is stopped, or the message flow is redeployed.
5. On the FTP tab, select the Remote Transfer property if you want the node to read files from an FTP or SFTP server using the following properties:
   1. In Transfer Protocol, specify the protocol that is to be used for remote file transfer. Possible values are FTP and SFTP.
   2. In Remote server and port, supply the IP address and port number of the FTP or SFTP server to be used. Use one of the following syntaxes:
      • IP_address_or_URL
      • IP_address_or_URL:port_number

      If you specify the IP address in IPv6 format, ensure that you enclose it in square brackets, as in the following examples:

      • [12a::13bd:24cd]
      • [12a::13bd:24cd]:123 where 123 is the port number

      If you are using FTP and you do not specify a port number, 21 is assumed. If you are using SFTP and you do not specify a port number, a port number of 22 is assumed. However, if an FTP Server policy is defined, you can enter the name of the policy in this field (see FTP Server policy (FtpServer)).
   3. In Security identity, specify the name of a security identity that has been defined using the mqsisetdbparms command. The user identifier and password that are to be used to log on to the FTP or SFTP server are obtained from this definition, the name of which must have an ftp:: prefix. The value of this property is overridden by the value in the Security identity property of the FTP Server policy, if it is set.
   4. In Server directory, specify the directory in the FTP or SFTP server from which to transfer files. The default is a period (.) which means the default directory after logon. If you specify a relative path, the directory is based on the default directory after FTP or SFTP logon. Ensure that the syntax of the path conforms to the file system standards in the FTP or SFTP server. The value in this property is overridden by the value in the Remote FTP server directory property of the FTP Server policy, if it is set.

      If you select both the Remote Transfer and Include local subdirectories properties, only the subdirectories on the local system are searched for files to process. Whereas on the remote system, only the top-level directory that you specify is searched for files to process.

   5. In Scan delay, specify the delay, in seconds, between directory scans. The default is 60 seconds. The value set in this property overrides the value set for the polling interval on the Polling tab when the Remote Transfer property is selected. The value of this property is overridden by the value in the Scan delay property of the FTP Server policy, if it is set.
   When using FTP, the LocalEnvironment propagated from the FileExists node looks similar to the following example:

   0x01000000:Name):Files     = (
     (0x03000000:NameValue):TimeStamp    = '20220327_092919_509715' (CHARACTER)
     (0x03000000:NameValue):ProcessedBy  = 'File Exists' (CHARACTER)
     (0x03000000:NameValue):FromFTP     = TRUE (BOOLEAN)
     (0x03000000:NameValue):Server      = 'ourFTPServer.hursley.ibm.com:21' (CHARACTER)
     (0x03000000:NameValue):TotalInList = 1 (INTEGER)
     (0x01000000:Name):File     = (
       (0x03000000:NameValue):Directory    = 'C:\Work\Development\V12\FileNodes\Exists' (CHARACTER)
       (0x03000000:NameValue):ServerDirectory = '/home/mbfvftp/FTP_Node/RemoteFiles' (CHARACTER)
       (0x03000000:NameValue):Name         = 'InputMsg.xml' (CHARACTER)
       (0x03000000:NameValue):LastModified = GMTTIMESTAMP '2022-01-26 11:26:32.025' (GMTTIMESTAMP)
       (0x03000000:NameValue):IsEmpty      = FALSE (BOOLEAN)
       (0x01000000:Name):Wildcard = (
         (0x03000000:NameValue):WildcardMatch = 'InputMsg.xml' (CHARACTER)
       )
     )
   )

   When using FTP, only the Server and ServerDirectory fields are included in the LocalEnvironment. These are the only FTP fields that can be overridden on the FileRead node. If a FileRead node is used together with a FileExists node, the Remote Transfer, Transfer protocol and Server identity properties of the FileRead node must be configured to match those of the FileExists node.

6. On the Transactions tab, set the transaction mode. Although all file operations are non-transactional, the transaction mode on this node determines whether the rest of the nodes in the flow are to be run under sync point. Select Yes if you want the flow updates to be treated transactionally, if possible, or No if you do not. The default for this property is No.
7. Optional: On the Instances tab, set values for the properties that control the additional instances (threads) that are available for a node. For more information, see Configurable properties in a BAR file.