Considerations for a Large Number of Files in a Watch Directory
After Sterling Connect:Direct File Agent scans a watch directory, it submits a Sterling Connect:Direct Process for each available file that it finds. If it locates a large number of files in a watch directory, the time required to submit the Processes is larger than if it were handling a few files during each scan. Also, watch directories with a large number of files is not scanned as often because of the time required for a scan.
Also, consider that the Sterling Connect:Direct Server must be able to process the Processes that Sterling Connect:Direct File Agent submits. A limit may exist on how many Processes it runs concurrently. IBM tested with over 50,000 files in a watch directory with no adverse effects. The checkpoint file contained over 50,000 entries and files transferred appropriately. An upper limit to the files in a watch directory has not been determined, and depends on the amount of memory the system has available.