Filtering patterns and examples

Filtering patterns and examples that demonstrate the effects of adding more filter rules to the command and show how to format a filter rule file.

Note: You can synchronize Windows, Linux, macOS, and other Unix-based endpoints and servers, but must take care with path separators. The path separator "/" is supported on Windows and other platforms. The path separator "\" is platform independent only for the options -d/r/L/R/B/b and --keep-dir-local/remote. In Aspera Sync filtering rules, however, "\" is exclusively a quoting operator and "/" is the only path separator recognized.

Command line syntax

Basic usage

  • Filtering rules are applied to the transfer list in the order that they appear on the command line. If filtering rules are configured in aspera.conf, they are applied before the rules on the command line.
  • --include="pattern". Include paths that match pattern. Wildcards, such as * (asterisk) and ? (question mark), are supported but rules that contain them must be in double quotation marks. For example, --include="*.jpg". For more information on how to set include and exclude patterns.
  • --exclude="pattern". Exclude paths that match pattern. Wildcards, such as * (asterisk) and ? (question mark), are supported but rules that contain them must be in double quotation marks. For example, --exclude="*.jpg".
  • An include rule must be followed by at least one exclude rule, otherwise all files are transferred because none are excluded.
  • Filtering operates only on the set of files and directories in the transfer list.

    For more information about async commands, see Async command reference.

Rule patterns

Rule patterns (globs) use standard globbing syntax that includes wildcards and special characters, and several Aspera extensions to the standard.

  • Character case: Case always matters, even if the file system does not enforce such a distinction. For example, on Windows FAT or NTFS file systems and macOS HPFS+ a file system search for DEBUG returns files Debug and debug. In contrast, sync filter rules use exact comparison, such that debug does not match Debug. To match both, use [Dd]ebug.
  • Partial matches: With globs, unlike standard regular expressions, the entire file name, or directory name must match the pattern. For example, the pattern abc*f matches abcdef but not abcdefg.

Standard globbing: Wildcards and special characters

/ The only recognized path separator.
\ Quotes any character literally, including itself. \ is exclusively a quoting operator, not a path separator.
* Matches zero or more characters, except / or the . in /..
? Matches any single character, except / or the . in /..
[] Matches exactly one of a set of characters, except / or the . in /..
[^] When ^ is the first character, matches exactly one character not in the set.
[!] When ! is the first character, matches exactly one character not in the set.
[x-x] Matches exactly one of a range of characters.
[:xxxxx:] For details about this type of wildcard, see any POSIX-standard guide to globbing.

Globbing Extensions: Wildcards and special characters

no / or * at end of pattern Matches files only.
/ at end of pattern Matches directories only. With -N, no files under matched directories or their subdirectories are included in the transfer. All subdirectories are still included, although their files are not included. However, with -E, excluding a directory also excludes all files and subdirectories under it.
* or /** at end of pattern Matches both directories and files.
/** Like * but also matches / and the . in /..
/ at start of pattern Must match the entire string from the root of the transfer set. (Note: The leading / does not refer to the system root or the docroot.)

Standard globbing examples

Wildcard Example Matches Does Not Match
/ abc/def/xyz abc/def/xyz abc/def
\ abc\? abc? abc\? abc/D abcD
* abc*f abcdef abc.f abc/f abcefg
? abc?? abcde abc.z abcdef abc/d abc/.
[] [abc]def adef cdef abcdef ade
[^] [^abc]def zdef .def 2def bdef /def /.def
[!] [!abc]def zdef .def 2def cdef /def /.def
[:xxxxx:] [[:lower:]]def cdef ydef Adef 2def .def

Globbing extension examples

Wildcard Example Matches Does Not Match
/** a/**/f a/f a/.z/f a/d/e/f a/d/f/ za/d/f
* at end of rule abc* abc/ abcfile  
/** at end of rule abc/** abc/.file abc/d/e/ abc/
/ at end of rule abc/*/ abc/dir abc/file
no / at end of rule abc abc (file) abc/
/ at start of rule /abc/def /abc/def xyz/abc/def

Examples

  1. Include files under top-level directories Raw and Jpg.
    # async ... --include='/Raw/**' --include='/Jpg/**' --exclude='*' \ 
    --exclude='.*' ...  
  2. Same as Example 1, except also include directories that start with ".", at any level.
    # async ... --include='.*/' --include='/Raw/**' --include='/Jpg/**' \ 
    --exclude='*' --exclude='.*' ...  
  3. Same as Example 2, except exclude regular files that end in "~" or ".thm".
    # async ... --include='.*/' --exclude='.*~' --exclude='*~' \
    --exclude='.*.thm' --exclude='*.thm' --include='/Raw/**' \
    --include='/Jpg/**' --exclude='*' --exclude='.*' ...  
  4. Same as Example 3, except include only certain directories under Jpg.
    # async ... --exclude='.*~' --exclude='*~' --exclude='.*.thm' \
    --exclude='*.thm' --include='.*/' --include '/Raw/**' \
    --include='/Jpg/Big/**' --include='/Jpg/Med/**' \
    --exclude='*' --exclude='.*' ...  

    The long sequence in Example 4 can also be represented as a file:

    # async ... --exclude-from=- <<EOF
    # no regular files with ~ suffix, dot or otherwise:
        .*~
        *~
    # similarly for ".thm" suffix files:
        .*.thm
        *.thm
    # include directories starting with "."
        + .*/  
    # include everything else found under top-level Raw :
        + /Raw/**
    # and under Big/ and Med/ in Jpg:
        + /Jpg/Big/**
        + /Jpg/Med/**
    # but nothing else:
        *
        .*
    EOF