Creating pull Watch Folders in the GUI

The GUI enables you to easily create Watch Folders that pull files and directories from a remote server to a local directory as they are added to a remote directory.

Restrictions on all Watch Folders
  • Only local to remote (push) and remote to local (pull) configurations are supported. Remote to remote and local to local are not supported.
  • Growing files are only supported for local sources (push Watch Folders) and must be authenticated by a transfer user (password or SSH key file). The transfer user cannot be restricted to aspshell and the source cannot be in object storage. Growing files are only supported for local sources and nonobject storage-based destinations.
  • Source file archiving is not supported if the Watch Folder source is in object storage.
  • IBM Aspera Shares endpoints must have version Shares version 1.9.11 with the Watch Folder patch or a later version.
Restrictions on Pull Watch Folders
  • The remote server must be running HSTS or HSTE.
  • Pull Watch Folders must be authenticated with an access key ID and secret, a Node API username and password, SSH keys, SSH username and password, or IBM Aspera Shares credentials. (V4.4.3)
  • Pull Watch Folders that use Node API authentication cannot be authenticated with a Node API user whose associated transfer user is configured with a restriction (the Watch Folder status is reported as impaired). Edit the transfer user's configuration to use a docroot, restart asperanoded, and the Watch Folder recovers automatically.
  • Pull Watch Folders cannot use IBM Aspera on Cloud (including IBM Aspera on Cloud transfer service nodes) or IBM Aspera Transfer Cluster Manager nodes as the remote source.
  • Pull Watch Folders do not support growing files.
Note: While Watch Folder use involves a server-to-server transfer, the server on which the watch folder is configured is referred to as the client.
  1. Prepare the client as described in Getting started with Watch Folders in the GUI.
  2. Create a Watch Service on the remote server.
    If you have SSH access to the server, create the service from the server's command line.
    1. Create the service.
      $ /Library/Aspera/sbin/asperawatchd --user username
      The username is for a system user with permissions to the source path.
    2. Confirm that the service was created.
      $ /Library/Aspera/bin/aswatchadmin query-daemons

      If the service exists, the following output is returned (in this example, the user is "root"):

      $ /Library/Aspera/bin/aswatchadmin query-daemons
      [aswatchadmin query-daemons] Found a single daemon:
           root

      If other services are running on the server, other daemons are also returned.

    If you do not have SSH access to the server, use the Node API from your local computer to create the service. This approach requires that you have node credentials for the server. For instructions, see Creating a pull Watch Folder with the API.
  3. To create a Watch Folder, click Create a new Watch Folder.
    If the error message, "You cannot create Watch Folders. Please contact your Administrator." is displayed, the Node API user is not configured with the necessary permissions. Node API user permissions can be modified as described in Configuring custom Watch Folder permissions policies in the GUI. To configure a Node API user with all admin permissions, run the following command:
    $ /Library/Aspera/bin/asnodeadmin -a -u node_username -p node_password -x transfer_user --acl-set "admin,impersonation"
  4. Configure Watch Folder settings.
    1. Watch Folder Service: If no Watch Folder services exist, one is created for the transfer user that is associated with Node API username that was used for login. To create a Watch Folder service under a different user, click Create and follow the substeps in step 2. If a service exists for the transfer user, it is automatically populated. To run the Watch Folder under a different user or service, click Change and select the correct user and service combination.
    2. Watch Folder name: A unique name for the Watch Folder.
    3. Watchd scan period: Set the amount of time between assessments of the watch (from end of one to start of the next).
      Important: For pull Watch Folders, file systems scans that are triggered by the scan period interval are the sole means for detecting changes in the source directory. Shorter scan periods detect changes faster but can result in greater resource consumption, particularly for object storage. For most use cases, a 1-minute scan period balances detection frequency with resource consumption.

      The scan period can be specified with units, such as 30 m for 30 minutes, or 24-hour clock, such as 01:00:00 for 1 hour. Watchd assesses watches for change independent of the snapshot minimum interval and snapshot minimum changes to ensure that changes are captured.

    4. Advanced: Click Advanced to enter the ID of an existing Watch service, rather than creating a new service.
    5. Direction: Select Pull to transfer from the remote server to the local computer.
    6. Target path: Click Browse to select the target path.
    7. Host (and authentication): The IP address, DNS, hostname, or URL of the remote server. Click Import to import connection information from the Connections list. The username, authentication, and target path are automatically populated from the connection settings, as are settings under Transfer and File Handling.
      If you are entering the host manually, use the following syntax based on the type of remote endpoint and authentication method:
      • HSTS or HSTE authenticated with Node API or access key credentials: Enter the node URL as https://ip_address_or_server_url:9092/. If a different HTTPS port is configured, replace 9092 with the correct port. Enter the Node API username or access key ID as the User and the Node API user's password or the access key secret as the Password.
      • IBM Aspera Shares: Enter the URL of the Shares server as https://ip_address:443 and provide the Shares login credentials.
      Note: Pull Watch Folders must be authenticated with an access key ID and secret, a Node API username and password, or Shares credentials. SSH authentication is not supported for remote sources. If using node credentials, the transfer user that is associated with the Node API user must have a doc root that is configured, not a restriction.
    8. Source path: Click Browse to select the local source path.
    9. Configure other Watch Folder settings.
      To ensure that only one drop is created for each scan interval, go to Settings and set the Drops Detection cool off to a value greater than the Watchd scan period.

      For information about all Watch Folder settings, see Watch Folder configuration reference.

  5. Once all required fields are set and any other configuration is done, click OK to create the Watch Folder.
    If the source directory contains files, the Watch Folder collects them into a drop after the Watch service scan interval passes and transfers them to the target.
    Note: No files are transferred until the first scan interval passes. If the Watch service scan interval is set to the default, files transfer after 30 minutes.

    If the transfer does not start after the scan period, see Troubleshooting Watch Folders.

    When you create a Watch Folder, a Watch service subscription is automatically created to monitor the source directory. In the rare case that the subscription is somehow deleted or impaired, Watch Folders automatically creates a new subscription; however, the new subscription does not retain the file change history and all files in the source directory are retransferred.