Operator BWListTagger

SPL standard and specialized toolkits > com.ibm.streams.cybersecurity 2.1.0 > com.ibm.streams.cybersecurity.tagging > BWListTagger

The BWListTagger operator loads lists of domains and IP addresses into a lookup table. When an IP or domain is ingested, the operator checks if it appears in one of the lookup tables. If it does, the operator tags the IP or domain as either "blackList" or "whiteList", depending on which table it appeared in. If the IP or domain does not appear in either table, it tags it as "nonMatched".

The operator is capable of dynamically updating (either adding to or removing from) the lookup tables via a control port. To update the lookup tables, submit a tuple of type com.ibm.streams.cybersecurity.types::BWListUpdate_t to the control port. Once all updates have been submitted, a window punctuation must be submitted to the control port in order to apply the changes.

The operator supports both IPv4 and IPv6 addresses. For IPv4 addresses, the input attribute type can be either rstring, uint32, or list<uint32>. For IPv6 addresses, the input attribute type can be either rstring, list<uint8>[16] or list<list<uint8>[16].

Behavior in a consistent region

The operator is not supported in a consistent region. A warning occurs when you compile your streams processing application.
The operator cannot be the start of a consistent region. An error occurs when you compile your streams processing application.

Exceptions

This operator will log an error if an attempt is made to load an invalid IP address or domain into the lookup tables.

Summary

Ports: This operator has 2 input ports and 1 output port.
Windowing: This operator does not accept any windowing configurations.
Parameters: This operator supports 7 parameters.
Optional: blackDomainFile, blackIPFile, domainAttr, domainSearchAlgo, ipAttr, whiteDomainFile, whiteIPFile
Metrics: This operator does not report any metrics.

Properties

Implementation: C++
Threading: Never - Operator never provides a single threaded execution context.

Input Ports

Ports (0)

Ingests tuples that have attributes that contain domain or IP addresses.

Properties

Optional: false

Ports (1)

This port can be used to add or remove IPs and domains stored in the lookup tables. This port ingests tuples of type com.ibm.streams.cybersecurity.types::BWListUpdate_t. Updates submitted via this port are not immediately applied to the internal lookup tables. Instead, updates are stored in a temporary staging area and do not get added to the primary lookup tables until this port receives a window punctuation. When a punctuation is received, processing on the data input port (port 0) is blocked until the operator has finished updating the internal lookup tables. Once the update is complete, processing will resume.

Properties

Optional: true

Output Ports

Assignments: This operator allows any SPL expression of the correct type to be assigned to output attributes.

Output Functions

TaggerFunctions

<any T> T AsIs()

The default function for output attributes. By default, this function assigns the output attribute to the value of the input attribute with the same name.

list<enum{nonMatched,whiteList,blackList}> getDomainTags()

Returns the black and white list tags for each domain in the domainAttr parameter. If the domainAttr parameter refers to an input attribute of type rstring, then this output function returns a type of BWListTag_e. If the domainAttr parameter refers to an input attribute of list<rstring>, then this output function returns a type of list<BWListTag_e>. The possible enum values that are returned from this function are: nonMatched, whiteList and blackListDomain.

list<enum{nonMatched,whiteList,blackList}> getIPTags()

Returns the black and white list tags for each IP in the ipAttr parameter. If the ipAttr parameter refers to an input attribute of type rstring, then this output function returns a type of BWListTag_e. If the ipAttr parameter refers to an input attribute of list<rstring>, then this output function returns a type of list<BWListTag_e>. The possible enum values that are returned from this function are: nonMatched, whiteList and blackListDomain.

list<enum{nonMatched,whiteList,blackList}> getAllTags()

Ports (0)

Submits a tuple for each input tuple received. Custom output functions are used to specify the value of the output tuple attributes. The output tuple attributes whose assignments are not specified are assigned from the input attribute.

Properties

Optional: false

TupleMutationAllowed: true
WindowPunctuationOutputMode: Generating

Parameters

Optional: blackDomainFile, blackIPFile, domainAttr, domainSearchAlgo, ipAttr, whiteDomainFile, whiteIPFile

blackDomainFile

A path to the file containing a list of black-listed domains. This file should contain a single domain per line. Relative paths are relative to the data directory. This parameter should not be used if input port 2 is defined.

Properties

Cardinality: 1
Optional: true
ExpressionMode: AttributeFree

blackIPFile

A path to the file containing a list of black-listed IP addresses. This file should contain a single IPv4 address per line. Relative paths are relative to the data directory. This parameter should not be used if input port 1 is defined.

Properties

Type
Cardinality: 1
Optional: true
ExpressionMode: AttributeFree

domainAttr

Refers to an expression containing domains to be tagged. The type of the expression can be either an rstring or a list<rstring>.

Properties

Optional: true
ExpressionMode: Expression

domainSearchAlgo

Specifies the algorithm to use when searching for domains in the lookup table. Valid values are exact and iterative. When exact is specified, the operator will try to find an exact match for the domain in the lookup domain. For example, assume the domain "ibm.com" is added to the domain whiteList. In this case, only incoming domains with the value "ibm.com" will be tagged. Any other domains (i.e. "us.ibm.com") will be tagged as "nonMatched".

When iterative is specified, the operator will continuously remove labels from the from of the domain until a match is found. This is useful if you want to add a second-level domain to the lookup table and have the operator tag domains with any number of labels. For example, assume the domain "ibm.com" is added to the domain whiteList. In this case, all of the following domains will be tagged as "whiteList": 'ibm.com', 'streams.ibm.com', 'a.b.c.ibm.com'.

Properties

Type: DomainSearchAlgoEnum (exact, iterative)
Cardinality: 1
Optional: true
ExpressionMode: CustomLiteral

ipAttr

Refers to one or more expressions containing IPs to be tagged. The type of the expression can be one of the following: rstring, list<rstring>, uint32, list<uint32>, list<uint8>[16], list<list<uint8>[16]>

Properties

Type
Optional: true
ExpressionMode: Expression

whiteDomainFile

A path to the file containing a list of white-listed domains. This file should contain a single domain per line. Relative paths are relative to the data directory. This parameter should not be used if input port 3 is defined.

Properties

Cardinality: 1
Optional: true
ExpressionMode: AttributeFree

whiteIPFile

A path to the file containing a list of white-listed IP addresses. This file should contain a single IPv4 address per line. Relative paths are relative to the data directory. This parameter should not be used if input port 4 is defined.

Properties

Cardinality: 1
Optional: true
ExpressionMode: AttributeFree

Libraries

No description for library.: Library Name: csa, re2; Library Path: ../../impl/lib/csa/, ../../impl/lib/re2/; Include Path: ../../impl/include/re2/, ../../impl/include/csa/, ../../impl/include/csa/bwtagging/, ../../impl/include/