Filter operator examples

Use the Filter operator to remove tuples from the streams flow according to a specific condition. The Filter operator acts as an ‘if’ statement.

The Filter operator is based on a limited subset of Python expressions. As a result, other Python expressions in this operator causes compilation errors.

The following Python expressions can be applied to specific data types.

Expression	Applicable data types
`>`	Number, date
`<`	Number, date
`==`	Number, date, text, boolean
`!=`	Number, date, text, boolean
`>=`	Number, date
`<=`	Number, date
`and`	Number, date, text, boolean
`or`	Number, date, text, boolean
`not`	Number, date, text, boolean
`is None`	Text
`is not None`	Text
`+, -, *, /`	Number
`x // y`	Number
`abs(x)`	Number
`int(x)`	Number
`float(x)`	Number
`pow(x, y), x ** y`	Number
`round(number[, ndigits])`	Number
`math.floor(x)`	Number
`math.ceil(x)`	Number
`str.endswith(suffix[, start[, end]])`	Text
`str.find(sub[, start[, end]])`	Text
`len(str)`	Text
`str.lower()`	Text
`str.replace(old, new)`	Text
`str.rfind(sub[, start[, end]])`	Text
`str.split(sep)`	Text
`str.startswith(prefix[, start[, end]])`	Text
`str(x)`	Number, date, boolean
`str.upper()`	Text
`str[from:to]`	Text
`datetime.datetime(y,m,d,h,mi,s)`	Date
`dt.day`	Date
`dt.month`	Date
`dt.year`	Date
`dt.hour, minute, second`	Date
`dt.microsecond`	Date
`dt.timestamp()`	Date
`dt.isoformat()`	Date

Text data types: Put strings in double quotation marks (“).

Delimiters: Double quotation marks (“), single quotation marks (‘), comma (,)

Boolean literals: True, False

Multiple lines and comments: To extend filter expressions over multiple lines, place the expression in parentheses. Then, you can also use comments. Comments begin with a hash mark (#) and extend until the end of line. See Example 5.

For more information about Python expressions, see Standard built-in types and Basic date and time types.

For information about date formats supported in streams flows, see Date formats.

Example 1

Task: You need to compare some date with a given date.

To compare a date to a given date, the syntax is datetime.datetime(y, m, d, h, mi, s). The date-time is in ISO 8601 format without time zone because the date must be in Coordinated Universal Time (UTC) time zone.

The Filter operator has the code eventTime > datetime.datetime(2017, 03, 13, 12, 5, 0).

Example 2

Task: You need to select tuples based on complex or nested selections.

Select tuples where (the UserID is Lee AND the date of the event is after March 13, 2017 at 12:05 PM) OR (the latitude is less than 40.5 AND the longitude is greater than 70.0).

The Filter operator has the code (userId == "Lee" and eventTime > datetime.datetime(2017, 3, 13, 12, 5, 0)) or (latitude < 40.5 and longitude >= 70).

Example 3

Task: You need to select tuples (‘Large’, ‘Medium’, ‘Small’) based on a range of values.

The Filter operator uses the following code.

    Predicates: {
      Large=i>1000,
      Medium=i>10 and i<1000,
      Small=i<10 };
      

If you use a pattern of Large Small, then a sequence of tuples where i has the values of 2000, 0 is a match. Conversely, where i has the values of 2000, 100, 10 is not a match, because the value 100 does not fit the pattern of Small.

Example 4

Task: You must convert non-string values to strings in order to concatenate them.

There is no automatic conversion of non-string values to string values when you concatenate them.

The code prefix + str(number) == a27 is valid. The code prefix + number == a27 is not valid.

Example 5

Task: You want to add a comment at the beginning of a code block. You also want to add comments within the code.