Metrics access
Metrics are simple counters that are maintained at run time, which can be read from outside of a running job to monitor statistics.
Two types of metrics are provided by the IBM® Streams instance, namely system and custom metrics. System metrics are predefined and are maintained by the IBM Streams instance. Custom metrics are created and maintained by the operators.
System metrics
System metrics come in two varieties: operator-level metrics and processing element-level metrics. Operator-level metrics are accessed through the getMetrics() member function of the OperatorContext class, which returns an object of type OperatorMetrics. The operator context object is accessible through the getContext() member function of the operator.
Individual operator-level metrics are available through the getInputPortMetric(port, metric) and getOutputPortMetric(port, metric) methods of the OperatorMetrics class, where port is the port index and metric is the name of the metric (of OperatorMetrics::InputPortMetricName and OperatorMetrics::OutputPortMetricName type). The available metrics include:
Name | Scope | Description |
---|---|---|
nTuplesProcessed | Per input port | The number of tuples that were processed. |
nTuplesDropped | Per input port | The number of tuples that were dropped. |
nTuplesQueued | Per input port | The number of tuples that are queued. |
nWindowPunctsProcessed | Per input port | The number of window punctuations that were processed. |
nFinalPunctsProcessed | Per input port | The number of final punctuations that were processed. |
nWindowPunctsQueued | Per input port | The number of window punctuations that are queued. |
nFinalPunctsQueued | Per input port | The number of final punctuations that are queued. |
queueSize | Per input port | The size of the queue for a threaded port, or 0 if the port is not threaded. |
nTuplesSubmitted | Per output port | The number of tuples that were submitted by the operator. |
nWindowPunctsSubmitted | Per output port | The number of window punctuations that were submitted by the operator. |
nFinalPunctsSubmitted | Per output port | The number of final punctuation that were submitted by the operator. |
Processing element-level metrics are accessed through the getMetrics() member function of the ProcessingElement class, which returns an object of type PEMetrics. The processing element object is accessible through the getPE() member function of the operator.
Individual processing element-level metrics are available through the getInputPortMetric(port, metric) and getOutputPortMetric(port, metric) methods of the PEMetrics class, where port is the port index and metric is the name of the metric (of PEMetrics::InputPortMetricName and PEMetrics::OutputPortMetricName type).
The Metric class is used to represent a metric. Each metric has a short name, a description, and a value of type int64. System metrics can be read, but cannot be written.
The available metrics include:
Name | Scope | Description |
---|---|---|
nTuplesProcessed | Per input port | The number of tuples that were processed. |
nWindowPunctsProcessed | Per input port | The number of window punctuations that were processed. |
nFinalPunctsProcessed | Per input port | The number of final punctuations that were processed. |
nTupleBytesProcessed | Per input port | The number of tuple bytes that were processed. |
nTuplesSubmitted | Per output port | The number of tuples that were submitted. |
nWindowPunctsSubmitted | Per output port | The number of window punctuations that were submitted. |
nFinalPunctsSubmitted | Per output port | The number of final punctuations that were submitted. |
nTupleBytesSubmitted | Per output port | The number of tuple bytes that were submitted. |
nBrokenConnections | Per output port | The number of broken connections that were detected. |
nRequiredConnecting | Per output port | The number of required connections that are currently connecting. |
nOptionalConnecting | Per output port | The number of optional connections that are currently connecting. |
nTuplesTransmitted | Per output port | The total number of tuples that were transmitted to all connected processing elements. |
nTupleBytesTransmitted | Per output port | The total number of tuple bytes that were transmitted to all connected processing elements. |
nConnections | Per output port | The number of processing elements that are connected to this output port. |
In addition to the processing element-level metrics described in Table 2, processing element-level connection metrics are provided by the system. The available metrics include:
Name | Scope | Description |
---|---|---|
congestionFactor | Per connection on each PE output port | An integer value (0 - 100) that represents the relative congestion for the connection. 0 means no congestion; 100 means that the connection is extremely congested. |
nTuplesFilteredOut | Per connection on each PE output port | The number of tuples that failed to meet the filter criteria. |
Each processing element output port can be connected to zero or more input ports on other processing elements. The connection information can be accessed by using ProcessingElement::getOutputPortConnections(), which returns a list of information about each connection, including the congestion factor for this connection and the nTuplesFilteredOut metric for the connection. From SPL, the information can be returned by using the native function spl.utility::getPEOutputPortConnections().
If a processing element restarts, all system metrics are reset to their initial values.
Custom metrics
Custom metrics that are listed in the operator model are automatically instantiated by the IBM Streams instance.
The names of all the custom metrics can be found by calling the getCustomMetricNames() member function of the OperatorMetric class. The custom metrics can be accessed at run time by the getCustomMetricByName(name) member function of the OperatorMetric class. More metrics can be created explicitly at run time, by using the createCustomMetric(name, description, kind) member function. For both functions, the returned metric objects of type Metric are owned by the IBM Streams instance, but the operator can update their value freely. The value of the metric can be updated by the use of setValue(newValue) and incrementValue(incValue) functions.
If a processing element restarts, custom metrics are reset to their initial value unless they are explicitly reset by the operator from a previous checkpoint.
The following example shows the runtime APIs for creating and updating custom metrics.
// member variable in operator
Metric * numInterestingEvents_;
...
// create the metric
OperatorMetrics & opm = getContext().getMetrics();
numInterestingEvents_ = & opm.createCustomMetric("nInterestingEvents",
"Number of very interesting events seen so far", Metric::Counter);
...
// update the metric
numInterestingEvents_->setValue(numEvents);
// increment the value
numInterestingEvents_->incrementValue(10);