Regex Filter transform

Edit online

The Regex Filter transform filters messages in the data stream according to a regular expression (regex) pattern, which you can define. You also define the Regex Filter to either accept or deny incoming messages based on the regular expression. For example, if an incoming message contains the regular expression, and you define the Regex Filter to deny incoming messages based on the regular expression, the Regex Filter then discards any incoming messages that contain the regular expression.

To use the Regex Filter transform, you must know how to use regular expressions. The Oracle documentation about regular expressions is one source of reference information.

Important:
  • The use of complex regular expressions can result in increased usage of system resources.
  • Between each data stream and each of its subscribers, only one Regex Filter transform is supported. If you have multiple regular expressions, combine them in one Regex Filter transform.

Configuration values that you can update

For the Regex Filter transform, you can update the following field values in the Configure Filter transform window:
Inspect
Specifies whether, and at what stage, data packets in the data stream are to be inspected. For example, during transform processing, the data packets can be inspected by printing them to the Z Common Data Provider Data Streamer job log at the input stage, the output stage, or both stages.
You can choose any of the following values. The default value is None. To prevent the sending of large volumes of data to the Z Common Data Provider Data Streamer job log, use the default value, unless you are instructed by IBM® Software Support to change this value for troubleshooting purposes.
None
Specifies that data packets are not inspected.
Input
Specifies that data packets are printed to the Data Streamer job log before they are processed by the transform.
Output
Specifies that data packets are printed to the Data Streamer job log after they are processed by the transform.
Both
Specifies that data packets are printed to the Data Streamer job log both before and after they are processed by the transform.
Regex
Specifies one or more valid regular expressions. At least one regular expression must be defined for the Regex Filter transform. You can also select the check box for any of the following expression flags:
Case Insensitive
Enables case-insensitive matching, in which only characters in the US-ASCII character set are matched.

To enable Unicode-aware, case-insensitive matching, select both the Unicode Case flag and the Case Insensitive flag.

Comments
Permits white space and comments in the regular expression. In this mode, white space is ignored, and any embedded comment that starts with the number sign character (#) is ignored.
Dotall
Enables dotall mode in which the dot expression (.) matches any character, including a line terminator.
Multi Line
Enables multiline mode in which the caret expression (^) and the dollar sign expression ($) match immediately after, or immediately before, a line terminator or the end of the message.
Unicode Case
Enables Unicode-aware case folding.

To enable Unicode-aware, case-insensitive matching, select both the Unicode Case flag and the Case Insensitive flag.

Unix Lines
Enables UNIX lines mode in which the dot expression (.), the caret expression (^), and the dollar sign expression ($) are interpreted only as the line feed (LF) line terminator.
To define one or more regular expressions in the Regex field, complete the following steps:
  1. Type a regular expression in the Regex field, and optionally, select one or more check boxes to define the matching modes.
  2. To add another regular expression, click ADD REGEX, and repeat the previous step.
Filter Type
Specifies whether the filter keeps or discards incoming messages that contain the regular expression.
You can choose either of the following values. The default value is Accept.
Accept
Specifies that any messages that contain the regular expression are kept in the data stream.
Deny
Specifies that any messages that contain the regular expression are discarded from the data stream.

Examples

The following examples show how to define a Regular Expression (Regex) to filter data.

Example 1
Regex expression
^\$HASP373.+SMF30.+$
Filter transform configuration
Figure 1. Configure regex example 1
Configure regex example 1
Input data before Regex Filter
IEF404I BPXAS - ENDED - TIME=18.53.28
 $HASP373 SMF30 STARTED - INIT A - CLASS A
 $HASP373 ZWESISTC STARTED
 IEF403I ZWESISTC - STARTED - TIME=18.28.34
 $HASP100 ZWESISTC ON STCINRDR
 $HASP395 FTPD ENDED - RC=0000
 $HASP373 FTPD STARTED
 IEF403I SSHD - STARTED - TIME=18.15.53
Output data after Regex Filter
$HASP373 SMF30 STARTED - INIT A - CLASS A
Example 2
Regex expression
^(\$HASP373|IEF403I).+(SMF30|ZWESISTC).+$
Filter transform configuration
Figure 2. Configure regex example 2
Input data before Regex Filter
IEF404I BPXAS - ENDED - TIME=18.53.28
 $HASP373 SMF30 STARTED - INIT A - CLASS A
 $HASP373 ZWESISTC STARTED
 IEF403I ZWESISTC - STARTED - TIME=18.28.34
 IEF403I zwesistc initialization completed
 $HASP100 ZWESISTC ON STCINRDR
 $HASP395 FTPD ENDED - RC=0000
 $HASP373 FTPD STARTED
 IEF403I SSHD - STARTED - TIME=18.15.53
Output data after Regex Filter
$HASP373 SMF30 STARTED - INIT A - CLASS A
$HASP373 ZWESISTC STARTED
IEF403I ZWESISTC - STARTED - TIME=18.28.34
IEF403I zwesistc initialization completed