July 2024: APAR OA66572

OMEGAMON Data Broker

No changes to OMEGAMON® Data Broker.

OMEGAMON Data Connect

HTTP output: batching

The new batching configuration parameter enables sending multiple records per HTTP POST request. Batching offers higher throughput than the previous single-record-per-request behavior.

For example:

http:
  enabled: true
  endpoints:
    ep1:
      url: http://example.com:9997
      batching:
        enabled: true # Default: false (single record per request, as before)
        # The following parameters apply only when batching is enabled
        batch-size: 5000 # Default: 1000
        linger: 1s # Default: 250ms

If batching is enabled, then the output sends up to batch-size records per request, according to the following rules:

• If the number of records in the queue is at least batch-size, then the output sends a request containing batch-size records.
• If there are fewer than batch-size records in the queue, then the output waits for up to the duration specified by linger. If the number of records in the queue reaches batch-size within that duration, then the output sends a request containing batch-size records. Otherwise, if there are still fewer than batch-size records in the queue at the end of that duration, then the output sends a request containing however many records are in the queue.

Previously, the HTTP output always sent only a single record per request, which remains the default behavior.

HTTP output: compression

The new compression configuration parameter enables gzip compression of HTTP POST request bodies.

Compression is particularly useful when batching is enabled.

Example configuration:

http:
  enabled: true
  endpoints:
    ep1:
      url: http://example.com:9997
      compression: true # Default: false
      batching:
        enabled: true

HTTP output: custom request headers

The new headers configuration parameter enables you to add custom headers to HTTP POST requests.

For example, the following configuration:

http:
  enabled: true
  endpoints:
    ep1:
      url: http://example.com:9997
      headers: # Custom headers to add to each HTTP POST request
        - key: odp_header_key_1
          value: header_value_1
        - key: odp_header_key_2
          value: header_value_2

adds the following headers to each request:

odp_header_key_1: header_value_1
odp_header_key_2: header_value_2

Some analytics platforms use custom headers to determine how to process a request. For details, see the documentation for your analytics platform.

Prometheus output: custom metric expiration

The new metric-expiration configuration parameter enables you to control when OMEGAMON Data Connect expires (removes) metrics from the Prometheus endpoint.

metric-expiration applies per table:

connect:
  output:
    prometheus:
      enabled: true
      mappings:
        products:
          <product_code>:
            enabled: true
            tables:
              <table_name>:
                enabled: true
                metric-expiration: <seconds>
                metrics: ...

The metric-expiration parameter is optional. For data from OMEGAMON monitoring agents, the default metric expiration behavior is unchanged.