Operator FMPFilter

Specialized toolkits - release 4.3.1.0-prod20190605 > com.ibm.streams.timeseries 5.0.1 > com.ibm.streams.timeseries.modeling > FMPFilter

Primitive operator image not displayed. Problem loading file: ../../image/tk$com.ibm.streams.timeseries/op$com.ibm.streams.timeseries.modeling$FMPFilter.svg

The FMPFilter operator is an adaptive faded-memory polynomial filter. It can be used for tracking, smoothing, and outlier or anomaly detection.

The FMPFilter operator tracks the data movements and simultaneously predicts the next expected time series as a smoothed value. It also flags anomalous samples. It models data by using a polynomial that is continuously re-estimated given the current data. As such, it is an adaptive system, which can mimic data movement and predict expected behavior.

The FMPFilter operator can operate on univariate and vector input time series. The FMPFilter operator is a univariate operator and as such processes a vector time series as a parallel sequence of univariate data. The FMPFilter also support expanding time series, where the number of components in the input list can increase overtime.

Behavior in a consistent region

  • The FMPFilter operator is not supported in a consistent region. A warning occurs when you compile your streams processing application.
  • The operator cannot be the start of a consistent region. An error occurs when you compile your streams processing application.

Exceptions

The FMPFilter operator throws an exception in the following cases:

  • The memoryLength parameter value is 0.
  • The integration parameter value is 0.
  • The size of the input time series list exceeds the maxDimension parameter value, if it is specified.
Examples

Summary

Ports
This operator has 1 input port and 1 output port.
Windowing
This operator does not accept any windowing configurations.
Parameters
This operator supports 9 parameters.

Required: degree, inputTimeSeries

Optional: cleanFrom, inputTimestamp, integration, maxDimension, memoryLength, partitionBy, thresholdFactor

Metrics
This operator does not report any metrics.

Properties

Implementation
C++
Threading
Never - Operator never provides a single threaded execution context.

Input Ports

Ports (0)

This port consumes data for training and scoring against the model. The inputTimeSeries parameter specifies the name of the attribute on this port that contains the time series data. The accepted data types are float64 and list<float64>.

Properties

Output Ports

Assignments
This operator allows any SPL expression of the correct type to be assigned to output attributes.
Output Functions
FMPFilterCOF
<any T> T predictedTimeSeries()

Returns the predicted time series data values either as a vector time series of list<float64> data types or a univariate time series of float64 data types, depending on the input time series format.

<any T> T anomalousFlags()

Returns the values of generated flags, which indicate whether there are anomalous samples in the input time series. For example, if a time series data value deviates from the permissible range, it is anomalous. If the flag is true, then the sample is anomalous. If the flag is false, the sample is within the permissible range.

<any T> T variance()

Returns the value of variance, which is calculated for each output time series value by using the integration value which the algorithm calculated. The variation for the time series input at index n is stored at index n in the output.

<any T> T predictedTimestamp()

Returns the timestamp of the predicted timeseries values. If the input timestamp is irregular, then the function returns a timestamp value of 0, and a message is logged. The supported return data types are SPL::timestamp and uint64, based on the type of the inputTimestamp parameter.

<any T> T AsIs(T v)

The default function for output attributes. By default, this function assigns the output attribute to the value of the input attribute with the same name.

Ports (0)

This port submits a tuple that contains the next predicted value as determined by the FMPFilter algorithm. This port submits a tuple each time a prediction is made. Custom output functions are used to specify the value of the output tuple attributes. The output tuple attributes whose assignments are not specified are assigned from input attributes.

Properties

Parameters

This operator supports 9 parameters.

Required: degree, inputTimeSeries

Optional: cleanFrom, inputTimestamp, integration, maxDimension, memoryLength, partitionBy, thresholdFactor

cleanFrom

Specifies to direct the algorithm to start ignoring anomalous values when it updates the model. After the cleanFrom number of time series tuples are read and used for updating the model, the next incoming time series tuples are updated by ignoring the anomalous values.

Properties

degree

Specifies the degree of the filter algorithm. The supported values are 0u, 1u, 2u, 3u. The expected data type is uint32.

Properties

inputTimeSeries

Specifies the name of the attribute that contains the time series data in the input tuple. The expected data types are float64 and list<float64>.

Properties

inputTimestamp

Specifies the name of the attribute in the input stream that contains the timestamp values. The timestamp attribute value indicates the number of microseconds since UNIX epoch. The expected data type is uint32 or SPL::timestamp.

Properties

integration

Specifies the amount of data to be used to estimate variance. This parameter specifies the number of input time series samples that the algorithm must use to determine the variance for the incoming time series values. When the algorithm calculates the permissible variation values, the next incoming input time series values are compared against the predicted value. If the difference is more than the calculated allowable variance, that particular input time series is flagged as anomalous.

Properties

maxDimension

Specifies the triggers for the time series expansion mode. This parameter value specifies the maximum number of time series that can be included in the process in real time. If you do not specify this parameter, the number of time series, as estimated from the first tuple, is assumed to be constant. The expected data type is uint32.

Properties

memoryLength

Specifies the number of time series values to be accounted for by the internal logic of the algorithm for predicting the next time series sample (maximum amount of history). The expected data type is uint32.

Properties

partitionBy

Specifies the name of the attribute that contains the key values that are associated with the time series values in the input tuple.

Properties

thresholdFactor

Specifies the threshold for checking for anomalies. Any input series that varies beyond the specified threshold value is an anomaly.

Properties

Code Templates

FMPFilter

stream<${schema}> ${outputStream} = FMPFilter(${inputStream}) 
{
	param
		inputTimeSeries:		${timeSeriesExpression};
		degree: 			${degreeExpression};

	output
		${outputStream}: ${outputExpression};
}

Libraries

No description for library.
Library Name: tsatapi
Library Path: ../../../impl/lib
Include Path: ../../../impl/include