Configuring Content Collector - Regular expression syntax

Regular expressions are written in a formal language that can be interpreted by a regular expression processor. You can use special characters and character sets in regular expressions to configure rules or property mappings in task routes.

Characters in regular expressions match a single instance of themselves with these exceptions:

The special characters: \ [ ^ $ | ? * + ( )
Characters that are defined in character classes

A character class is a defined set of characters and is enclosed in brackets. Any character that is specified in a character class except the special characters \ ^ - ] adds that character to the possible matches for the character set.

A bracket expression can contain any of these elements:

Any combination of single characters: For example, [abc] matches any of the characters a, b, or c.
Character ranges: For example, [a-c] matches any single character in the range of a to c.
Negations: For example, [^a-c] matches any character that is not in the range of a to c.
Predefined character classes: For example, [[:lower:]] matches any lowercase character.
Escape characters: For example, [\^] matches ^.

The range of a character class also depends on the locale of the machine where you run the task route.

You can use the following operations to construct regular expressions:

Alternation

A vertical bar (|) separates alternatives.

Grouping

Parentheses are used to define the scope and precedence of the operators.

Lookaround

Lookaround constructs are also called zero-width assertions. They actually match characters, but return only the result match or no match. The lookaround constructs are lookahead and lookbehind.

Positive and negative lookahead

Use positive lookahead to match something that follows a given pattern without making the pattern part of the match: match(?=pattern). Use negative lookahead if you want to match something that is not followed by something else: match(?!pattern).

Lookahead is typically used to create the logical AND of two regular expressions, for example, if a password must contain a lowercase letter, an uppercase letter, no punctuation marks, and be at least 8 characters long, you could use the following expression to validate the password:

(?=.*[[:lower:]])(?=.*[[:upper:]])(?!.*[[:punct:]]).{8,}

Positive and negative lookbehind

Lookbehind has the same effect as lookahead but works backwards. Positive lookbehind matches something that is preceded by a given pattern: (?<=pattern)match. Negative lookbehind matches something that is not preceded by a given pattern: (?<!pattern)match.

Quantification

A quantifier, such as the question mark (?), the asterisk (*), or the plus sign (+) after a token, such as a character or group, specifies how often the element that precedes it is allowed to occur. The standard quantifiers in regular expressions are greedy, meaning they match as much as they can. For example, a pattern of /.*/ that is applied to abc/123/xyz_6/7 returns 123/xyz_6 instead of 123 because with greedy quantification, as many characters as possible are returned. To avoid this problem, you can specify a quantifier as lazy, which is also known as nongreedy, by putting a question mark after the quantifier. With lazy quantification, the expression tries the shortest match first.

You can combine these constructions to form complex expressions.

When you set up regular expressions for matching operations, you can use several modifiers to determine how a regular expression is interpreted:

i: Match a pattern regardless of the case.
m: Treat the string as multiple lines. In this mode, the caret and the dollar sign match the start or end of any line anywhere within the string.
s: Treat string as single line. In this mode, the period matches any character, even a newline character.
x: Extend your pattern's legibility by permitting whitespace and comments.

These are usually written as /modifier, even though the delimiter in question might not really be a slash. You can also use any of these modifiers within the regular expression itself by using a (?modifier) construct, for example:

(?i)car matches car and CAR.

Operators are evaluated in the following order:

Collation-related bracket symbols: [==] [::] [..]
Escaped characters: \
Character set (bracket expression): []
Grouping: ()
Quantifiers: * + ? {m,n}
Concatenation
Anchoring: ^ $
Alternation: |

Remember: The case of the character or character class matters in some cases.

Table 1. Regular expression syntax
Character or character class	Description	Example
\	The backslash escapes special characters so that they are treated as literals.	`\+` matches `+`
[ ]	The brackets enclose a character class.	`[abc]`
^	The caret matches at the start of the string to which the regular expression is applied. If a bracket expression begins with the caret, it matches the complement of the characters it contains (negation).	`^.` matches `a` in `abc/xyz` `[^x-z]` matches any character that is not in the range x to z
$	The dollar sign matches the end of the string to which the regular expression is applied.	`.$` matches `z` in `abc/xyz`
\|	The vertical bar is used for alternatives and matches either of its arguments. Parentheses are used to group alternatives.	`abc\|def\|xyz` matches `abc`, `def`, or `xyz` `abc\|(def\|xyz)` matches `abcdef` or `abcxyz`
?	The question mark makes the item that directly precedes it optional. It is also used to set up lazy quantification.	`abc?` matches `ab` or `abc`
*	The asterisk indicates zero or more occurrences of the item that directly precedes it.	`ab*c` matches `ac`, `abc`, `abbc`, or `abbbc`
+	The plus sign indicates one or more occurrences of the item that directly precedes it.	`ab+c` matches `abc`, `abbc`, or `abbbc`, but not `ac`
()	Parentheses are used to group the parts of the regular expression.	`gr(a\|e)y` matches `gray` or `grey`
-	The hyphen specifies a range of characters unless it is specified immediately after an opening bracket. In this case, it is used literally.	`[A-Za-z0-9]` matches any letter or digit
.	The period matches any single character.
(?=`pattern`)	This construct matches something that is followed by a given pattern.	`a(?=b)` matches the `a` and only the `a` in `cab`, but does not match `bath` or `bar`.
(?!`pattern`)	This construct matches something that is not followed by a given pattern.	`a(?<!b)` matches the `a` and only the `b` in `bath` or `bar`, but does not match the `a` in `cab`.
(?<=`pattern`)	This construct matches something that is prededed by a given pattern.	`(?<=a)b` matches the `b` and only the `b` in `cab`, but does not match `bed` or `debt`.
(?<!`pattern`)	This construct matches something that is not prededed by a given pattern.	`(?<!a)b` matches the `b` and only the `b` in `bed` or `debt`, but does not match the `b` in `cab`.
\<	This escape sequence matches a the start of a word.	`\<ton` matches `tons` but not `button`.
\>	This escape sequence matches a the end of a word.	`\>ton` matches `button` but not `tons`.
\Q...\E	This sequence matches the characters between \Q and \E literally.	`\Q+-/\E` matches `+-/`
\d, \w, and \s	These shorthand character classes match the digits 0 - 9, word characters (letters, digits, and the underscore), and white space. \d is equivalent to [:digit:]. \w is equivalent to [:word:]. \s is equivalent to [:space:].	`[\d\s]` matches a character that is a digit or white space
\D, \W, and \S	These shorthand character classes are negated versions of the character classes that match digits, word characters, or white space. \W is equivalent to ^ [:word:]. \S is equivalent to ^ [:space:].	`\D` matches a character that is not a digit
\b	This shorthand character class matches a word boundary (the start or end of a word) unless it is specified inside a character class. In this case, \b is a backspace character.	`.\b` matches `c` in `abc`
\B	This shorthand character class matches only when it is not at a word boundary.	`\B.\B` matches `y` in `xyz`
\A	This shorthand character class matches at the start of the string to which the pattern is applied.	`\A.` matches `a` in `abc`
\z	This shorthand character class matches the end of the string to which the pattern is applied.	`\z.` matches `c` in `abc`
{n}	This quantifier repeats the item that directly precedes it exactly `n` times, where `n` is an integer equal to or greater than 1.	`a{3}` matches `aaa`
{n,m}	This quantifier repeats the item that directly precedes it between `n` and `m` times, where `n` is an integer equal to or greater than 0 and `m` is an integer equal to or greater than `n`.	`a{2,4}` matches `aaaa`, `aaa`, or `aa`
{n,}	This quantifier repeats the preceding item at least `n` times, where `n` is an integer equal to or greater than 0.	`a{2,}` matches `aaaaa` in `aaaaabc`
[:alphanum:]	This character class matches any alphanumeric character.
[:alpha:]	This character class matches any alphabetic character.
[:blank:]	This character class matches any white space that is not a newline character.
[:cntrl:]	This character class matches any control character, for example, the newline character or the backspace character.
[:digit:]	This character class matches any decimal digit. [:digit:] is equivalent to \d.
[:graph:]	This character class matches any graphical character: alphanumeric or punctuation.
[:lower:]	This character class matches any lowercase character.
[:print:]	This character class matches any printable character: alphanumeric, punctuation, or space.
[:punct:]	This character class matches any punctuation character.
[:space:]	This character class matches any white space, such as the blank character, the newline character, or the tab character. [:space:] is equivalent to \s.
[:upper:]	This character class matches any uppercase character.
[:word:]	This character class matches any word character: letters, digits, and the underscore. [:word:] is equivalent to \w.
[:xdigit:]	This character class matches any hexadecimal digit.