Watch Folder configuration reference
When you create or edit a Watch Folder in the GUI, you must specify the Watch Folder service, the source path, Watch scan period, and the remote connection. Other settings can be left with their default values or configured to meet your requirements. The following tables describe available Watch Folder settings by tab.
Watch Folder
These settings configure the Watch Folder source and destination, and the connection to the remote host.
| Setting | Description | Default |
|---|---|---|
| Watch Folder service | The Watch Folder service that is defined by its user and service ID, that the Watch Folder uses. | The first Watch Folder service in the list of services alphabetized by username. |
| Watch Folder name | A unique name for the Watch Folder. The value that is specified in this field is added to the cookie reported by ascp. Optional. | None specified |
Watchd scan period |
How frequently the Watch Service scans the source path for file system changes. For file systems with file notifications (Linux®, Windows™, macOS), set to 30 minutes (30 m) to provide a backup to the notification system, or set to infinite to never scan.
On file systems without file notifications, such as object storage, mounted storage (NFS), and AIX® file system scans triggered by the scan period are used to detect file changes. In this case, set the scan period to frequently scan for changes. On operating systems that support file notifications (Linux, Windows, macOS), asperawatchd uses the file notifications as the primary means for detecting changes, and the scan period serves as a backup. In this case, the default value of 30 minutes is usually acceptable and no change is necessary. To never scan, and rely entirely on file notifications, set to For pull Watch Folders, file systems scans that are triggered by Lower scan periods detect changes faster but can result in greater resource consumption, particularly for object storage. |
30 m |
| Advanced | Click Advanced to enter the ID of an existing Watch Folder service, rather than creating a new service. | None specified |
| Direction | Select Push to transfer from the local computer to a remote server.Select Pull to transfer from the remote server to the local computer. | Push |
| Local path | Click Browse to select the local path. The local path is the source for push Watch Folders, and the target for pull Watch Folders. | None specified |
| Host | The IP address, DNS, hostname, or URL of the remote server. If you enter the host manually, use the following syntax based on the type of remote endpoint and authentication method:
|
None specified |
| User | For SSH, the system account username. For a Shares server, the Shares username. For a node URL host, the Node API username or access key ID. | None specified |
| Authentication | For SSH authentication, select Password and enter the user's password, or select Public Key and select the user's public key from the drop-down menu.
For Shares hosts, enter the Shares user's password. For node authentication, enter the Node API user's password or access key secret. |
Password |
| SSH Port (TCP) | The port to use for SSH connections. | 22 |
| FASP® port (UDP) | The port to use for UDP connections. | 33001 |
| SSH host key fingerprint | The SSH fingerprint of the remote server. Use a SSH fingerprint for security. If the fingerprint does not match that of the server, the transfer fails with the error "Remote host is not who we expected". For more information, see Installing and configuring the SSH Server (Configuring Transfer Server Authentication). | None specified |
| Connection timeout | How long to wait for a connection to respond before it fails. | 10 s |
| Proxy URL | When using the address of an IBM Aspera Proxy server. The proxy syntax is: dnat(s)://user:password@server:port |
None specified |
| Remote path | Click Browse to select the source or target directory on the remote server. The remote path must be within the user's docroot.
For access key authentication, the path is relative to the storage specified in the access key. |
None specified |
Settings
These settings define how Watch Folder watches the file system and groups new files that were added to the source folder into drops. Drops are groups of files that are transferred together in one session, post-processed together, and reported as a unit.
| Setting | Description | Default |
|---|---|---|
| Sample period | Period used to compute the current bandwidth. Used with queue_threshold to compute the amount of data that is pushed to ascp. |
2 s |
| Queue threshold | Watch Folders controls the amount of data that is pushed to ascp for transferring. When the capacity is reached, Watch Folders waits before pushing new data. This capacity is based on the effective bandwidth that is reported by ascp. | 5 s |
| Snapshot creation period | The interval during which Watch Folders groups new files in the source directory into a drop. All files in a drop are transferred in the same transfer session, post-processed together, and reported as a unit. Watch Folders uses asperawatchd to detect file system modifications, and continuously creates snapshots to compute the snapshot differential. A small value results in high-temporal resolution for detecting file system modifications, whereas a large value improves asperawatchd performance. Default: 3s. | 3 s |
| Detection strategy | The strategy that Watch Folders uses to create drops when new files are added to the source folder:
|
Cool off only |
| (Drops) Detection cool off | The time after the first new file is added to the source file during which any other new files are included in the same drop. This setting is only relevant if the detection strategy is Cool off only. Set the detection cool off to a multiple of the Snapshot creation period for predictable results. | 5 s |
| Maximum parallel ascp | The maximum number of concurrent ascp sessions that Watch Folders can start. | 10 |
| (Files) Detection cools off | How long the Watch Folder service waits for the files in the watched folder to stop changing (stabilize) before it takes a directory snapshot and creates a drop. Default: 5 s. | 3 s |
| Filters | Restrict the files that are included in the Watch Folder transfer by setting filters.
Click Setting an Include filter protects matching files from subsequent Exclude filters. Filters are applied in order. Watch Folders supports glob and Regex filters. The glob filter system is the same as Ascp; see Using filters to include and exclude files. Note: An include rule must be followed by at least one exclude rule, otherwise all files are transferred because none are excluded. To exclude all files that do not match the include rule, use
Click /** for glob or .* for Regex. |
None specified |
Transfer
These settings configure the ascp transfer sessions that transfer files in each drop.
| Setting | Description | Default | ||||||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Bandwidth policy |
|
Fair | ||||||||||||||||||||||||||||||||||||||||
| Target rate | The target transfer rate. Transfer at rates up to the specified target rate. This option accepts suffixes T for terabits/s, G for gigabits/s, M for megabits/s, K for kilobits/s, or B for bits/s. Decimals are allowed. If this option is not set by the client, the setting in the server's aspera.conf is used. If a rate cap is set in the local or server aspera.conf, the rate does not exceed the cap. | 10.00 Mbps | ||||||||||||||||||||||||||||||||||||||||
| Minimum rate | Attempt to transfer no slower than the specified minimum transfer rate. | 0 bps | ||||||||||||||||||||||||||||||||||||||||
| Transport Encryption |
Select the encryption cipher. Aspera supports three sizes of AES cipher keys (128, 192, and 256 bits) and supports two encryption modes, Cipher Feedback mode (CFB) and Galois Counter Mode (GCM). The GCM mode encrypts data faster and increases transfer speeds compared to the CFB mode, but the server must support and permit it.
Cipher rules The encryption cipher that you are allowed to use depends on the server configuration and the version of the client and server:
Cipher Values
Client/Server Cipher negotiation The following table shows which encryption mode is used depending on the server and client versions and settings:
|
AES-128 | ||||||||||||||||||||||||||||||||||||||||
| SSH host key fingerprint | The SSH fingerprint of the remote server. Use a SSH fingerprint for security. If the fingerprint does not match that of the server, the transfer fails with the error "Remote host is not who we expected". For more information, see Installing and configuring the SSH Server (Configuring Transfer Server Authentication). | None specified | ||||||||||||||||||||||||||||||||||||||||
| Token | If required, the token string. Not valid for use with growing files. | None specified | ||||||||||||||||||||||||||||||||||||||||
| Tags | Specify custom metadata in JSON format. The tags object is passed directly to the ascp session. | None specified | ||||||||||||||||||||||||||||||||||||||||
| Read block size | The read block size. | None specified (uses the value set in aspera.conf). | ||||||||||||||||||||||||||||||||||||||||
| Write block size | The write block size. | None specified (uses the value set in aspera.conf). | ||||||||||||||||||||||||||||||||||||||||
| Datagram size | The datagram size (MTU) for FASP. | None specified (uses the detected path MTU). | ||||||||||||||||||||||||||||||||||||||||
| Rex message size | The maximum size of a retransmission request. Maximum: 1440. | None specified | ||||||||||||||||||||||||||||||||||||||||
| Raw ascp options |
Specify ascp options and their arguments that are not yet available in Watch Folders to apply to Watch Folder transfers. To use raw options, they must be enabled in the client's aspera.conf by running the following command:
|
None specified | ||||||||||||||||||||||||||||||||||||||||
| Cookie | Each transfer session (drop) includes a cookie that provides information about the Watch Folder that initiates the transfer session and the drop. The cookie has the following format:
You can override the default values by specifying your own. |
None specified (use default values) | ||||||||||||||||||||||||||||||||||||||||
| Content protection | Enter a password to enable client-side encryption at rest. Files that are uploaded to the server are appended with a .aspera-env extension. To download and decrypt .aspera-env files from the server, the client must provide the password. For more information about client-side encryption at rest, see Client-Side Encryption-at-Rest (EAR).
To disable content protection, clear the checkbox. |
None specified (content protection is disabled) | ||||||||||||||||||||||||||||||||||||||||
| Number of retry attempts | How many times to try transferring a file before the file is marked as failed. | 3 | ||||||||||||||||||||||||||||||||||||||||
| Wait between retries | How frequently to retry file transfers. | 3 s | ||||||||||||||||||||||||||||||||||||||||
| Retry maximum for | If no bytes are transferred during the specified period and no file is completed, the drop and all remaining incomplete files in the drop are marked as failed. | 1 m |
File Handling
These settings configure how files and their attributes are handled on the source and destination.
| Setting | Description | Default |
|---|---|---|
| Resume policy | Specify if partial files are resumed. To always retransfer files, select Never. To resume conditionally, select from the following options:
When resuming file transfer is enabled, select an overwrite rule for when the same file exists at the destination. Click the drop-down menu for If a complete file already exists at the destination. |
Never |
| Preserve file UIDs and GIDs | Select to preserve the file owner user ID or group ID. | None selected |
| Preserve file timestamps | Select to preserve all file timestamps. Equivalent to enabling Preserve creation timestamps, Preserve modification timestamps, and Preserve access timestamps.
Note: Access, modification, and source access times cannot be preserved for node and Shares connections that are using cloud storage.
|
Not enabled |
| Preserve creation timestamps | Set creation time of the destination be set to that of the source. If the destination is a non-WindowsTM host, this option is ignored. | Not enabled |
| Preserve modification timestamps | Set the modification time of the destination file to that of the source. | Not enabled |
| Preserve access timestamps | Set the access time of the destination to that of the source. The destination file has the access time of the source file before the transfer. | Not enabled |
| Source Handling | Select what to do with source files after they are successfully transferred to the destination. Files can be archived, deleted, or retained after transfer of a drop. When files are archived or deleted, source subdirectories are also deleted from the source, unless the subdirectories were empty to start. File structure is preserved in the archive.
File archiving (Automatically move source files to an archive folder) is not supported for sources in object storage.
|
Do nothing after transfer |
Growing files
Growing files (files that are written to the source folder from a streaming input) are transferred by using ascp4 technology rather than ascp. Identify growing files by specifying filters; files that match the filters are considered growing files. This feature requires a growing files-enabled license.
These settings configure how growing files are identified and the transfer session.
| Setting | Description | Default |
|---|---|---|
| Maximum active drops | The maximum number of concurrent ascp4 sessions the Watch Folder can initiate. | 8 |
| Bandwidth policy |
|
Fair |
| Target rate | The target transfer rate. Transfer at rates up to the specified target rate. This option accepts suffixes T for terabits/s, G for gigabits/s, M for megabits/s, K for kilobits/s, or B for bits/s. Decimals are allowed. If this option is not set by the client, the setting in the server's aspera.conf is used. If a rate cap is set in the local or server aspera.conf, the rate does not exceed the cap. | 10.00 Mbps |
| Minimum rate | Attempt to transfer no slower than the specified minimum transfer rate. | 0 bps |
| Datagram size | The datagram size (MTU) for FASP. | None specified |
| Transport encryption | Select the encryption cipher (AES-128) to use for encrypting data in transit, or disable encryption by selecting none. | AES-128 |
| SSH Port (TCP) | The port to use for SSH connections. | 22 |
| FASP Port (UDP) | The port to use for UDP connections. | 33001 |
| Completion timeout | How long to wait before the session is considered complete. A growing file is considered complete when no new data arrives within the timeout period. | 5s |
| Send data after maximum | Force ascp4 stream to send data after the given time, even if the chunk is not full. | 2s |
| Memory | The maximum amount of memory FASPStream is allowed to use. | 2.00 MB |
| Chunk size | Packet size for transfers over the network. | 128.00 KB |
| Filters | Identify growing files as the ones that match the specified filters. Click Note: An include rule must be followed by at least one exclude rule, otherwise all files are transferred because none are excluded. To exclude all files that do not match the include rule, use
/** for glob or .* for Regex.Click |
None specified |
Packages
These settings enable the use of package files to define the order of files in the transfer queue for a session, and specify which file, either the last in the list or the package file itself, to transfer last. The package file is identified by setting matching filters.
| Setting | Description | Default |
|---|---|---|
| Package timeout | How long to wait for file dependencies to be satisfied (files that must be transferred before the last file are transferred) before you consider the dependency as unsatisfied. | 10s |
| Parsers | Click
Identify package files as the ones that match the specified filters. Click Note: An include rule must be followed by at least one exclude rule, otherwise all files are transferred because none are excluded. To exclude all files that do not match the include rule, use
/** for glob or .* for Regex.Click |
None specified |