Configuring filters to include and exclude files
For example, on Windows FAT or NTFS (or directories) to transfer by indicating which to skip or include based on name matching. When no filtering rules are specified by the client, ascp transfers all source files in the transfer list.
Filters can be specified on the ascp command line and in aspera.conf. Ascp applies filtering rules that are set in aspera.conf before it applies rules on the command line.
Set filtering rules in the GUI
- Click Configuration > File Handling.
- Scroll down to File Filter Pattern List.
- Select Override then click to open the filter Define dialog. If rules were added
earlier, either through the GUI or through aspera.conf, they appear in the window.
- Click to add a filtering rule, or click to delete a rule that you selected.
Rule syntax
A rule consists of a +
or -
sign that indicates whether to
include or exclude a rule, followed by a space character and by a pattern. A pattern can be a file
or directory name, or a set of names expressed with UNIX glob patterns.
+
or -
that are
part of rule syntax. The purpose of each is different and they are unrelated.Basic usage
- Filtering rules are applied to the transfer list in the order from top to bottom.
- Filtering is a process of exclusion, and the include rules overrides the exclude rules that follow them. Include rules cannot add back files that are excluded by a preceding exclude rule.
- Include rules must be followed by at least one exclude rule, otherwise all files are
transferred because none are excluded. To exclude all unmatched files, add two final rules:
"
- *
" and "- .*
". - Filtering operates only on the set of files and directories in the transfer list. An include rule cannot add files or directories that are not already part of the transfer list.
Example | Transfer Result |
---|---|
-
rule |
Transfer all files and directories except those with names that match rule. |
+
rule |
Transfer all files and directories because none are excluded. To transfer only the files and directories with names that match rule use:
|
+ rule1- rule2 |
Transfer all files and directories with names that match rule1, as well as all other files and directories except those with names that match rule2. |
- rule1+ rule2 |
Transfer all files and directories except those with names that match
rule1. All files and directories that are not already excluded by
rule1, are included because no additional exclude rule follows -N
rule2 . Additional filters can be set for transfers in the UI (Adding and editing connections) or with the command line (Using filters to include and exclude files).To transfer only the files and directories with names that do not match rule1 but do match rule2 use:
|
Filtering rule application
Filtering order
Filtering rules are applied to the transfer list in the order that they appear in the list.
Example
Consider the following set of rules:
+ file2
- file[0-9]
If the source contains file1, file2, and fileA, the filtering rules are applied as follows:
Rule patterns
Rule patterns (globs) use standard globbing syntax that includes wildcards and special characters, and several Aspera extensions to the standard.
- Character case: Case always matters, even if the file system does not enforce such a
distinction. For example, on Windows FAT or NTFS file systems and macOS HPFS+ a file system search
for
DEBUG
returns filesDebug
anddebug
. In contrast, ascp filter rules use exact comparison, such thatdebug
does not matchDebug
. To match both, use[Dd]ebug
. - Partial matches: With globs, unlike standard regular expressions, the entire file
name or directory name must match the pattern. For example, the pattern
abc*f
matchesabcdef
but notabcdefg
.
Standard globbing: Wildcards and special characters
/ |
The only recognized path separator. |
\ |
Quotes any character literally, including itself. \ is exclusively a
quoting operator, not a path separator. |
* |
Matches zero or more characters, except / or the .
in /. . |
? |
Matches any single character, except / or the . in
/. . |
[ … ] |
Matches exactly one of a set of characters, except / or the
. in /. . |
[^ … ] |
When ^ is the first character, matches exactly one character
not in the set. |
[! … ] |
When ! is the first character, matches exactly one character
not in the set. |
[ x- x] |
Matches exactly one of a range of characters. |
[: xxxxx:] |
For details about this type of wildcard, see any POSIX-standard guide to globbing. |
Globbing Extensions: Wildcards and special characters
no / or * at end of pattern |
Matches files only. |
/ at end of pattern |
Matches directories only. With -N , no files under matched directories
or their subdirectories are included in the transfer. All subdirectories are still included,
although their files are not included. However, with -E , excluding a directory also
excludes all files and subdirectories under it. |
* or /** at end of pattern |
Matches both directories and files. |
/** |
Like * but also matches / and the .
in /. . |
/ at start of pattern |
Must match the entire string from the root of the transfer set. (Note: The leading
/ does not refer to the system root or the docroot.) |
Standard globbing examples
Wildcard | Example | Matches | Does Not Match |
---|---|---|---|
/ |
abc/def/xyz |
abc/def/xyz |
abc/def |
\ |
abc\? |
abc? |
abc\? abc/D abcD |
* |
abc*f |
abcdef abc.f |
abc/f abcefg |
? |
abc?? |
abcde abc.z |
abcdef abc/d abc/. |
[ … ] |
[abc]def |
adef cdef |
abcdef ade
|
[^ … ] |
[^abc]def |
zdef .def 2def |
bdef /def /.def |
[! … ] |
[!abc]def |
zdef .def 2def |
cdef /def /.def |
[: xxxxx:] |
[[:lower:]]def |
cdef ydef |
Adef 2def .def |
Globbing extension examples
Wildcard | Example | Matches | Does Not Match |
---|---|---|---|
/** |
a/**/f |
a/f a/.z/f a/d/e/f |
a/d/f/ za/d/f |
* at end of rule |
abc* |
abc/ abcfile |
|
/** at end of rule |
abc/** |
abc/.file abc/d/e/ |
abc/ |
/ at end of rule |
abc/*/ |
abc/dir |
abc/file |
no / at end of rule |
abc |
abc (file) |
abc/ |
/ at start of rule |
/abc/def |
/abc/def |
xyz/abc/def |
Testing your filtering rules
You can use this procedure to test your filtering rules.
Example filter rules
These example rules are based on downloading a directory AAA from your server to C:\tmp\dest on your computer:
These examples use the following file set:
AAA/abc/def
AAA/abc/.def
AAA/abc/.wxy/def
AAA/abc/wxy/def
AAA/abc/wxy/.def
AAA/abc/wxy/tuv/def
AAA/abc/xyz/def/wxy
AAA/wxyfile
AAA/wxy/xyx/
AAA/wxy/xyxfile
Key for interpreting results:
< xxx/yyy
= Excludedxxx/yyy
= Includedzzz/
= directory namezzz
= filename1- Transfer everything except files and directories that start with
".
":
+ *
- AAA/**
Results:
AAA/abc/def
AAA/abc/wxy/def
AAA/abc/wxy/tuv/def
AAA/abc/xyz/def/wxy
AAA/wxyfile
AAA/wxy/xyx/
AAA/wxy/xyxfile
< AAA/abc/.def
< AAA/abc/.wxy/def
< AAA/abc/wxy/.def
2- Exclude directories and files whose names start with wxy
- wxy*
Results:
AAA/abc/def
AAA/abc/.def
AAA/abc/.wxy/def
AAA/abc/xyz/def/
< AAA/abc/wxy/def
< AAA/abc/wxy/.def
< AAA/abc/wxy/tuv/def
< AAA/abc/xyz/def/wxy
< AAA/wxyfile
< AAA/wxy/xyx/
< AAA/wxy/xyxfile
3- Include directories and files that start with wxy
if they fall directly
under AAA:
+ wxy*
- AAA/**
Results:
AAA/wxy/
AAA/wxyfile
< AAA/abc/def
< AAA/abc/.def
< AAA/abc/.wxy/def
< AAA/abc/wxy/def
< AAA/abc/wxy/.def
< AAA/abc/wxy/tuv/def
< AAA/abc/xyz/def/wxy
< AAA/wxy/xyx/
< AAA/wxy/xyxfile
4- Include directories and files at any level that start with wxy, but do not include dot-files, dot-directories, or any files under the wxy directories, unless they start with wxy. However, subdirectories under wxy are included:
+ */wxy*
- AAA/**
Results:
AAA/abc/wxy/tuv/
AAA/abc/xyz/def/wxy
AAA/wxyfile
AAA/wxy/xyx/
< AAA/abc/def
< AAA/abc/.def
< AAA/abc/.wxy/def
< AAA/abc/wxy/def *
< AAA/abc/wxy/.def
< AAA/abc/wxy/tuv/def
< AAA/wxy/xyxfile
* Even though wxy is included, def is excluded because it's a file.
5- Include wxy directories and files at any level, even those starting
with ".
":
+ */wxy*
- */wxy/**
- AAA/**
Results:
AAA/abc/wxy/def
AAA/abc/wxy/.def
AAA/abc/wxy/tuv/def
AAA/abc/xyz/def/wxy
AAA/wxyfile
AAA/wxy/xyx/
AAA/wxy/xyxfile
< AAA/abc/def
< AAA/abc/.def
< AAA/abc/.wxy/def
6- Exclude directories and files that start with wxy, but only those found at a specific location in the tree:
+ /AAA/abc/wxy*
Results:
AAA/abc/def
AAA/abc/.def
AAA/abc/.wxy/def
AAA/abc/xyz/def/wxy
AAA/wxyfile
AAA/wxy/xyx/
AAA/wxy/xyxfile
< AAA/abc/wxy/def
< AAA/abc/wxy/.def
< AAA/abc/wxy/tuv/def
7- Include the wxy directory at a specific location, and include all its
subdirectories and files, including those starting with ".
":
+ AAA/abc/wxy/**
- AAA/**
Results:
AAA/abc/wxy/def
AAA/abc/wxy/.def
AAA/abc/wxy/tuv/def
< AAA/abc/def
< AAA/abc/.def
< AAA/abc/.wxy/def
< AAA/abc/xyz/def/wxy
< AAA/wxyfile
< AAA/wxy/xyx/
< AAA/wxy/xyxfile