Native histogram metric support (public preview)

Native histogram metric support is a public preview feature that is available for all tenant units in the IBM Instana Observability (SaaS) offering.

IBM Instana Observability now supports the storage and retrieval of Prometheus histogram metrics as a native metric type.

Storage formats of Prometheus histogram metrics

In IBM Instana Observability, the histogram metrics are stored as a native metric type. The Java, .NET Core, and Prometheus sensors support the storage of Prometheus histograms as native histogram metrics.

In earlier releases of IBM Instana Observability, each part of a histogram was treated as a separate metric. For example, the following Prometheus histogram is displayed as seven individual metrics in curated dashboards:

vehicle_repair_history_bucket{category="EV",deductible="100.00",le="2.0",} 13.0
vehicle_repair_history_bucket{category="EV",deductible="100.00",le="4.0",} 13.0
vehicle_repair_history_bucket{category="EV",deductible="100.00",le="6.0",} 13.0
vehicle_repair_history_bucket{category="EV",deductible="100.00",le="8.0",} 13.0
vehicle_repair_history_bucket{category="EV",deductible="100.00",le="+Inf",} 13.0
vehicle_repair_history_count{category="EV",deductible="100.00",} 13.0
vehicle_repair_history_sum{category="EV",deductible="100.00",} 13.0

With the native Prometheus histogram metric support, the preceding histogram is referred to holistically as follows:

  • In curated dashboards: vehicle_repair_history{category="EV",deductible="100.00"}
  • In custom dashboards (with the category and deductible labels available for filtering and grouping results): vehicle_repair_history

The names of histogram metrics that are natively stored do not have any metric type identifiers, such as _bucket, _sum, or _count, and they do not contain bucket labels, such as le="2.0".

You can use the following widgets to view the descriptive statistics of a histogram metric:

  • Histogram widget: Displays the distribution of measurements for the selected time window.
  • Chart: Time Series widget: Displays the percentiles, sum, or mean.

You can visualize a histogram in curated or custom dashboards.

Visualizing in curated dashboards

In curated dashboards, the mean value is shown for natively stored histograms. The mean value for a time point is the average of measurements that are recorded in the histogram metric since the last time point. The lowest metric granularity that is available for histograms is 10 seconds.

The following dashboard example shows the mean delta value for the natively stored vehicle_repair_history histogram.

Curated dashboard histogram metrics

Visualizing in custom dashboards

You can visualize natively stored histograms on custom dashboards by using the following widgets:

Visualizing by using the Histogram widget

To visualize natively stored histograms by using the Histogram widget, complete the following steps:

  1. Go to the Instana UI to create a custom dashboard. For more information about how to create a custom dashboard, see Create a Custom Dashboard.

  2. In the custom dashboard, click Add Widget and select the Histogram widget type. The Add Widget - Histogram dashboard is displayed.

    Histogram Widget

  3. In the Metric selection section, select Infrastructure & Platforms from the Data Source list. Infrastructure & Platforms is the only data source that is available for visualization as a histogram.

  4. From the Metric list, select a histogram metric that corresponds to a natively stored histogram.
    The following example shows that several Prometheus histogram metric names that begin with vehicle_repair. However, only the first histogram metric (vehicle_repair_history) is a natively stored histogram. So, you must select it.

    Histogram Widget datasource selection

    Note: When you are selecting a metric to visualize using the Histogram widget, ensure that you select a metric name that corresponds to a natively stored histogram (otherwise the results might be unpredictable). You can determine which metric names correspond to natively stored histograms by using the following criteria:

    • The metric type is histogram.
    • The metric name does not list label values in braces ({ }) following the name.
    • The metric name does not contain metric type identifiers, such as _bucket, _sum, or _count, and it does not include bucket labels, such as le="2.0".

  5. Optional: Select the filter criteria. For example, you can filter the histogram data using one or more Prometheus histogram metric labels.

  6. Select the metric value formatter. The default formatter is Percentage, which might not be appropriate for the histogram metric.
    In the following example, the histogram data is filtered by using the deductible label, and Number is selected from the Formatter list in the Axis configuration section. The number formatter is ideal in this example because the histogram buckets are whole numbers.

    Histogram Widget Filter

  7. In the Preview section, preview the visualization result as histograms.

    Histogram preview

The histogram bucket values differ from the original metric values that are reported by Prometheus because the histogram visualization is based on the differences between observations rather than absolute cumulative values.

Visualizing by using the Time Series widget

To visualize natively stored histograms by using the Chart: Time Series widget, complete the following steps:

  1. Go to the Instana UI to create a custom dashboard. For more information, see Create a Custom Dashboard.

  2. In the custom dashboard, click Add Widget then select the Chart: Time Series widget type.

  3. On the Add Widget - Chart: Time Series dashboard, select the Infrastructure & Platforms from the Data Source list.

  4. From the Metric list, select a natively stored histogram metric. For more information about how to select the correct histogram metric, see the note in step 4 in the Visualizing by using the Histogram widget section.

  5. Optional: Select the aggregation, filter, and group criteria. In the following example, the 99th percentile aggregation is selected, and the histogram data is grouped by using the category label.

    Time Series histogram configuration

  6. In the Preview section, preview the visualization result as histograms.

    Time Series histogram preview

Important: When you are using the Chart: Time Series widget to visualize natively stored histograms, consider the following points:

  • Maximum and minimum values are not available for Prometheus histograms.
  • Percentiles are only available if the histogram has a minimum of two finite bucket boundaries.
  • For a particular time point, if no measurements are recorded for a histogram since the previous time point, then no data is displayed at that time point. As a result, gaps might appear in the time series graph.

Limitations of natively stored histograms

Natively stored histograms have the following limitations:

  • The lowest metric granularity that is available for natively stored histograms is 10 seconds.
  • Prometheus histogram metrics require more than one observation to be usable for calculations. As such, measurements from the histogram before the first observation do not affect the data that is displayed.
  • For a particular time point, if no measurements are recorded for a histogram since the previous time point, then no data is displayed at that time point. If a histogram metric never had measurements, then it is not available as a natively stored histogram.