Operator IPFIXMessageParser

Specialized toolkits - release 4.3.1.0-prod20190605 > com.ibm.streamsx.network 3.2.1 > com.ibm.streamsx.network.parse > IPFIXMessageParser

Primitive operator image not displayed. Problem loading file: ../../image/tk$com.ibm.streamsx.network/op$com.ibm.streamsx.network.parse$IPFIXMessageParser.svg

IPFIXMessageParser is an operator for the IBM Streams product that parses individual fields of IPFIX messages received in input tuples, and emits tuples containing message data. The operator may be configured with one or more output ports, and each port may be configured to emit different tuples, as specified by output filters. The tuples contain individual fields from the input message, as specified by output attribute assignments.

The IPFIXMessageParser operator expects only IPFIX messages in its input tuples, without any of the headers that precede them in network packets. The PacketLiveSource and PacketFileSource operators can produce tuples that contain IPFIX messages with the PAYLOAD_DATA() output attribute assignment function.

The IPFIXMessageParser operator consumes input tuples containing IPFIX messages, parses indivdual fields in flow records within IPFIX messages, selects flows to emit as output tuples with filter expressions, and assigns values to them with output attribute assignment expressions.

Output filters and attribute assignments are SPL expressions. They may use any of the built-in SPL functions, and any of these functions, which are specific to the IPFIXMessageParser operator:

There are result functions for each of the standardized fields. Nonstandardized fields in flow records can be accessed by field number.

The IPFIXMessageParser operator emits a tuple on each output port for each flow record within a IPFIX message, optionally filtered by the 'outputFilters' parameter. Specified fields from each flow record are assigned to output attributes with the IPFIX parser result functions. All attributes of all output ports must be assigned values, either with explicit assignment expressions, or implicitly by copy from input tuples.

This operator is part of the network toolkit. To use it in an application, include this statement in the SPL source file: 


use com.ibm.streamsx.network.parse::*;

Sample Applications

The network toolkit includes several sample applications that illustrate how to use this operator.

References

IPFIX messages and the fields they contain are described here:

The result functions that can be used in boolean expressions for the outputFilters parameter and in output attribute assignment expressions are described here:

Summary

Ports
This operator has 1 input port and 0 or more output ports.
Windowing
This operator does not accept any windowing configurations.
Parameters
This operator supports 4 parameters.

Required: messageAttribute, sourceAttribute

Optional: outputFilters, processorAffinity

Metrics
This operator does not report any metrics.

Properties

Implementation
C++
Threading
Always - Operator always provides a single threaded execution context.

Input Ports

Ports (0)

The IPFIXMessageParser operator requires one input port. One input attribute must be of type blob and must contain a IPFIX message, excluding the network headers that proceed them in network packets, as specified by the required parameter messageAttribute.

The PAYLOAD_DATA() output assignment function of the PacketLiveSource and PacketFileSource operators produces attributes that can be consumed by the IPFIXMessageParser operator.

Properties

Output Ports

Assignments
This operator allows any SPL expression of the correct type to be assigned to output attributes.
Ports (0...)

The IPFIXMessageParser operator requires one or more output ports.

Each output port will produce one output tuple for each input tuple if the corresponding expression in the outputFilters parameter evaluates true, or if no outputFilters parameter is specified.

Output attributes can be assigned values with any SPL expression that evaluates to the proper type, and the expressions may include any of the IPFIX result functions. Output attributes that match input attributes in name and type are copied automatically.

Properties

Parameters

Required: messageAttribute, sourceAttribute

Optional: outputFilters, processorAffinity

messageAttribute

This required parameter specifies an input attribute of type blob that contains a IPFIX message to be parsed by the operator.

Properties

outputFilters

This optional parameter takes a list of SPL expressions that specify which IPFIX messages should be emitted by the corresponding output port. The number of expressions in the list must match the number of output ports, and each expression must evaluate to a boolean value. The output filter expressions may include any of the IPFIX result functions.

The default value of the outputFilters parameter is an empty list, which causes all IPFIX messages processed to be emitted by all output ports.

Properties

processorAffinity

This optional parameter takes one expression of type uint32 that specifies which processor core the operator's thread will run on. The maximum value is P-1, where P is the number of processors on the machine where the operator will run.

Where the operator runs on a thread of its own, this parameter applies to the operator's thread. This is the situation when the operator's input port is configured as a threaded input port, and when the operator has an @parallel annotation.

Where the operator runs on the thread of an upstream operator, this parameter affects the thread of the operator that sends tuples to it. This is the situation when the operator is fused with an upstream operator.

The default is to dispatch the operator's thread on any available processor.

Properties

sourceAttribute
Properties

Libraries

common code for toolkit operators
Include Path: ../../impl/include