IBM Streams 4.2.1

Operator CrossCorrelate

SPL standard and specialized toolkits > com.ibm.streams.timeseries 4.0.1 > com.ibm.streams.timeseries.analysis > CrossCorrelate

Primitive operator image not displayed. Problem loading file: ../../image/tk$com.ibm.streams.timeseries/op$com.ibm.streams.timeseries.analysis$CrossCorrelate.svg

DEPRECATED: The com.ibm.streams.timeseries.analysis.CrossCorrelate operator is deprecated and is replaced by the com.ibm.streams.timeseries.analysis.CrossCorrelate2 operator. The deprecated operator might be removed in a future release.

In signal processing, cross-correlation is a measure of similarity of two time series as a function of a time-lag applied to one of them. Cross-correlation is also known as a sliding dot product or sliding inner-product, which is defined by the following formula: (f*g)[n]=Σ f*[m] g[n+m]

In this formula, f and g are two input time series. Cross-correlation is often used in pattern recognition. Output of cross-correlation indicates the amount of overlap or similarities between two input time series.

The CrossCorrelate operator is a window-based operator. The CrossCorrelate operator inserts tuples from each input port into the corresponding window. When both the windows are full, the CrossCorrelate operator calculates the cross-correlation between the tuples by using the Fast Fourier transformation or by using the standard cross–correlation formula: (f*g)[n]=∑ f*[m] g[n+m].

Behavior in a consistent region

  • The CrossCorrelate operator is not supported in a consistent region. A warning occurs when you compile your streams processing application.
  • The operator cannot be the start of a consistent region. An error occurs when you compile your streams processing application.

Exceptions

The CrossCorrelate operator throws an exception and terminates when a count-based window configuration of 0 is specified.

Windowing

The CrossCorrelate operator supports a tumbling window with count-based, delta-based, time-based, or punctuation-based eviction policy.

The tuples from each input port are stored in the corresponding window. When both the windows are full, the cross-correlation is calculated for tuples in both the windows and the windows are flushed. The cross-correlated time series is output followed by a window marker punctuation.

When a time-based window or a punctuation-based window with no tuples tumbles, only a window marker punctuation is output.

When one of the input ports receives the final punctuation marker, the CrossCorrelate operator stores the tuples in memory. When the window corresponding to the other input port tumbles, the CrossCorrelate operator calculates the cross-correlation between these tuples and the tuples that are stored in memory. When both the input ports receive the final punctuation marker, the CrossCorrelate operator calculates the cross-correlation for the tuples in both these windows and outputs the cross-correlated times series to the output port.

Examples

The following example shows how you can use the CrossCorrelate operator to find correlation between two audio time series:

use com.ibm.streams.timeseries.analysis::CrossCorrelate;

composite Main {
  stream<float64 data > ts1 = FileSource() {
    param
      file   : "tsList0.dat";
      format : csv;
  }
  stream<float64 data > ts2 = FileSource() {
    param
      file : "tsList1.dat";
      format:csv;
  }
  stream <list<float64> correlatedTimeSeries> Out1 = CrossCorrelate(ts1; ts2) {
    window
      ts1 : tumbling,count(10);
      ts2 : tumbling,count(5);
    param
      inputTimeSeries : ts1.data,ts2.data;
      algorithm       : FFT;
    output
      Out1: correlatedTimeSeries = crossCorrelateTimeSeries();
  }
  () as out1Sink = FileSink(Out1) {
    param
      file   : "Result-Out1.dat";
      format : csv;
  }
}

Summary

Ports
This operator has 2 input ports and 1 output port.
Windowing
This operator requires a windowing configuration.
Parameters
This operator supports 3 parameters.

Required: inputTimeSeries

Optional: algorithm, normalize

Metrics
This operator does not report any metrics.

Properties

Implementation
C++
Threading
Never - Operator never provides a single threaded execution context.

Input Ports

Ports (0)

This port consumes the values of the first timeseries to be cross correlated. The inputTimeSeries parameter specifies the name of the attribute on this port that contains the time series data. The accepted data type is float64.

Properties

Ports (1)

This port consumes the values of the second timeseries to be cross correlated. The inputTimeSeries parameter specifies the name of the attribute on this port that contains the time series data. The accepted data type is float64.

Properties

Output Ports

Assignments
This operator allows any SPL expression of the correct type to be assigned to output attributes.
Output Functions
CrossCorrelatefunctions
<any T> T AsIs(T v)

The default function for output attributes. By default, this function assigns the output attribute to the value of the input attribute with the same name.

list<float64> crossCorrelateTimeSeries()

This function returns a list<float64> value that holds the cross-correlated time series data. If the input time series in port 0 is of size n and input time series in port 1 is of size m, the size of the cross-correlated time series is calculated by using the formula max(m,n)-min(m,n).

Ports (0)

This port submits a tuple that contains the result of the cross correlation. This port submits a tuple each time the two timeseries are cross correlated, which occurs when both input port windows are full. Custom output functions are used to specify the value of the output tuple attributes. The output tuple attributes whose assignments are not specified are assigned from input attributes.

Properties

Parameters

This operator supports 3 parameters.

Required: inputTimeSeries

Optional: algorithm, normalize

algorithm

Specifies the algorithm the CrossCorrelate operator uses to calculate the cross-correlation. The supported values are FFT and standard. If FFT is specified, the CrossCorrelate operator calculates the cross-correlation between two time series by using the Fast Fourier Transform algorithm. The default value is set to standard.

Properties

inputTimeSeries

This parameter specifies the name of the input time series attributes from the two input ports. The attribute expression in index 0 maps to port 0 and the attribute expression in index 1 maps to port 1. The supported type is float64.

Properties

normalize

Specifies whether the operator calculates raw or normalized cross correlation. The operator normalizes the output time series by using the following formula: ncorr(x,y)= corr(x,y)/sqrt(corr(x,x)*corr(y,y)), where x and y represent the input time series. The default value is set to false.

Properties

Libraries

Cross correlate library
Library Name: tsatapi
Library Path: ../../../impl/lib
Include Path: ../../../impl/include