IBM Streams 4.2.1

Metrics access

Metrics are simple counters that are maintained at run time, which can be read from outside of a running job to monitor statistics.

Two types of metrics are provided by the IBM® Streams instance, namely system and custom metrics. System metrics are predefined and are maintained by the IBM Streams instance. Custom metrics are created and maintained by the operators.

System metrics

System metrics come in two varieties: operator-level metrics and processing element-level metrics. Operator-level metrics are accessed through the getMetrics() member function of the OperatorContext class, which returns an object of type OperatorMetrics. The operator context object is accessible through the getContext() member function of the operator.

Individual operator-level metrics are available through the getInputPortMetric(port, metric) and getOutputPortMetric(port, metric) methods of the OperatorMetrics class, where port is the port index and metric is the name of the metric (of OperatorMetrics::InputPortMetricName and OperatorMetrics::OutputPortMetricName type). The available metrics include:

Table 1. Operator-level metrics provided by the system
Name Scope Description
nTuplesProcessed Per input port The number of tuples that were processed.
nTuplesDropped Per input port The number of tuples that were dropped.
nTuplesQueued Per input port The number of tuples that are queued.
nWindowPunctsProcessed Per input port The number of window punctuations that were processed.
nFinalPunctsProcessed Per input port The number of final punctuations that were processed.
nWindowPunctsQueued Per input port The number of window punctuations that are queued.
nFinalPunctsQueued Per input port The number of final punctuations that are queued.
queueSize Per input port The size of the queue for a threaded port, or 0 if the port is not threaded.
nTuplesSubmitted Per output port The number of tuples that were submitted by the operator.
nWindowPunctsSubmitted Per output port The number of window punctuations that were submitted by the operator.
nFinalPunctsSubmitted Per output port The number of final punctuation that were submitted by the operator.

Processing element-level metrics are accessed through the getMetrics() member function of the ProcessingElement class, which returns an object of type PEMetrics. The processing element object is accessible through the getPE() member function of the operator.

Individual processing element-level metrics are available through the getInputPortMetric(port, metric) and getOutputPortMetric(port, metric) methods of the PEMetrics class, where port is the port index and metric is the name of the metric (of PEMetrics::InputPortMetricName and PEMetrics::OutputPortMetricName type).

The Metric class is used to represent a metric. Each metric has a short name, a description, and a value of type int64. System metrics can be read, but cannot be written.

The available metrics include:

Table 2. Processing element-level metrics provided by the system
Name Scope Description
nTuplesProcessed Per input port The number of tuples that were processed.
nWindowPunctsProcessed Per input port The number of window punctuations that were processed.
nFinalPunctsProcessed Per input port The number of final punctuations that were processed.
nTupleBytesProcessed Per input port The number of tuple bytes that were processed.
nTuplesSubmitted Per output port The number of tuples that were submitted.
nWindowPunctsSubmitted Per output port The number of window punctuations that were submitted.
nFinalPunctsSubmitted Per output port The number of final punctuations that were submitted.
nTupleBytesSubmitted Per output port The number of tuple bytes that were submitted.
nBrokenConnections Per output port The number of broken connections that were detected.
nRequiredConnecting Per output port The number of required connections that are currently connecting.
nOptionalConnecting Per output port The number of optional connections that are currently connecting.
nTuplesTransmitted Per output port The total number of tuples that were transmitted to all connected processing elements.
nTupleBytesTransmitted Per output port The total number of tuple bytes that were transmitted to all connected processing elements.
nConnections Per output port The number of processing elements that are connected to this output port.

In addition to the processing element-level metrics described in Table 2, processing element-level connection metrics are provided by the system. The available metrics include:

Table 3. Processing element-level connection metrics provided by the system
Name Scope Description
congestionFactor Per connection on each PE output port An integer value (0 - 100) that represents the relative congestion for the connection. 0 means no congestion; 100 means that the connection is extremely congested.
nTuplesFilteredOut Per connection on each PE output port The number of tuples that failed to meet the filter criteria.

Each processing element output port can be connected to zero or more input ports on other processing elements. The connection information can be accessed by using ProcessingElement::getOutputPortConnections(), which returns a list of information about each connection, including the congestion factor for this connection and the nTuplesFilteredOut metric for the connection. From SPL, the information can be returned by using the native function spl.utility::getPEOutputPortConnections().

If a processing element restarts, all system metrics are reset to their initial values.

Custom metrics

Custom metrics that are listed in the operator model are automatically instantiated by the IBM Streams instance.

The names of all the custom metrics can be found by calling the getCustomMetricNames() member function of the OperatorMetric class. The custom metrics can be accessed at run time by the getCustomMetricByName(name) member function of the OperatorMetric class. More metrics can be created explicitly at run time, by using the createCustomMetric(name, description, kind) member function. For both functions, the returned metric objects of type Metric are owned by the IBM Streams instance, but the operator can update their value freely. The value of the metric can be updated by the use of setValue(newValue) and incrementValue(incValue) functions.

If a processing element restarts, custom metrics are reset to their initial value unless they are explicitly reset by the operator from a previous checkpoint.

The following example shows the runtime APIs for creating and updating custom metrics.

// member variable in operator
Metric * numInterestingEvents_;
...
// create the metric
OperatorMetrics  & opm = getContext().getMetrics();
numInterestingEvents_ = & opm.createCustomMetric("nInterestingEvents",
    "Number of very interesting events seen so far", Metric::Counter);
...
// update the metric
numInterestingEvents_->setValue(numEvents);
// increment the value
numInterestingEvents_->incrementValue(10);