HTTP output parameters

In the context of OMEGAMON Data Connect sending data over HTTP, OMEGAMON Data Connect is the client and the destination is the server.

connect:
  output:
    http:
      enabled: <true|false> # Default at this level: false
      endpoints: # One or more endpoints (destinations)
        <endpoint_name_1>: # Each endpoint has a unique name of your choice
          enabled: <true|false> # Default at this level: true
          url: <string>
          # All of the following parameters are optional
          call-timeout: <seconds>
          connect-timeout: <seconds>
          write-timeout: <seconds>
          read-timeout: <seconds>
          max-failures: <number>
          compression: <true|false> # Default: false
          headers: # One or more custom headers to add to each request
            - key: <header_key_1>
              value: <header_value_1>
            - key: <header_key_2>
              value: <header_value_2>
          batching: # Send multiple records per request
            enabled: <true|false> # Default: false (single record per request)
            # batch-size and linger apply only when batching is enabled
            batch-size: <number_of_records> # Default: 1000
            linger: <duration> # Default: 250ms
          ssl:
            <SSL parameters>
          filter: # Output-level filter
            <Filter parameters>
        <endpoint_name_2>: # Additional endpoint
          ...

enabled

Whether this function is enabled. Allowed values: true, false. This key is optional.

You can specify an enabled key in various contexts. The default value depends on the context. Example defaults:

• connect.output.http.enabled: false
• connect.output.http.endpoints.<endpoint_name>.enabled: true
• connect.output.http.endpoints.<endpoint_name>.batching.enabled: false

Specifying enabled: false has the same effect as commenting-out the parent key of the enabled key and all descendants of that parent key.

To enable sending to any endpoints, you must specify connect.output.http.enabled: true.

To disable sending to an endpoint, specify connect.output.http.endpoints.<endpoint_name>.enabled: false.

To disable sending to all endpoints, either omit connect.output.http.enabled or specify connect.output.http.enabled: false.

<endpoint_name_1>, <endpoint_name_2>, ...

OMEGAMON Data Connect can send to multiple endpoints.

Endpoint names are your choice. You might choose descriptive names, such as instana. See the examples at the end of this topic.

url

HTTP or HTTPS URL of the destination endpoint. If the URL does not specify a port, then the default port is 80 for HTTP or 443 for HTTPS.

If you specify an HTTPS URL, then you must also specify SSL parameters.

call-timeout

Optional. Either:

• Integer number of seconds to allow for a request to complete before considering the request a failure
• 0 (zero), meaning unlimited; no timeout

Default: 0 (no timeout).

The call timeout covers the entire duration of an HTTP request, ending when OMEGAMON Data Connect receives the complete response from the server. Other timeout parameters cover individual stages of a request.

A call timeout contributes to the failures counted by max-failures.

connect-timeout

Optional. Either:

• Integer number of seconds to allow for establishing a connection to the server before considering the request a failure
• 0 (zero), meaning unlimited; no timeout

Default: 10.

OMEGAMON Data Connect reuses connections. The connect timeout applies only to an HTTP request that requires a new connection.

write-timeout

Optional. Either:

• Integer number of seconds to allow for writing the request to the server before considering the request a failure
• 0 (zero), meaning unlimited; no timeout

Default: 10.

One possible cause of write timeout is an unusually large request body.

read-timeout

Optional. Either:

• Integer number of seconds to allow for reading a response from the server before considering the request a failure
• 0 (zero), meaning unlimited; no timeout

Default: 10.

The elapsed time to read a complete response can exceed the read timeout. A response can arrive in a single piece or in multiple pieces spread over time. The read timeout applies to each attempt to read a piece of the response.

Tip: There is no separate timeout parameter for the complete response. To indirectly set a timeout for the complete response, set call-timeout, which applies to the entire HTTP request, including the time to read the complete response.

max-failures

Optional. Maximum number of failures to allow before stopping sending requests to the endpoint. Default: no value; unlimited.

Possible failures include timeouts, connection failures, and unsuccessful responses.

To avoid unlimited failures, set a max-failures value. For example:

• To not allow any failures, and stop sending requests to the endpoint immediately after the first failure, set max-failures to 0 (zero)
• To allow 5 failures, and then stop sending requests to the endpoint if a 6th failure occurs, set max-failures to 5

compression

Optional. Whether to gzip-compress the request body. Allowed values: true, false. Default: false; the request body is not compressed.

headers

Optional. Sequence of custom header key-value pairs to add to each request.

For example, the following headers parameter:

headers:
  - key: odp_header_key_1
    value: header_value_1
  - key: odp_header_key_2
    value: header_value_2

adds the following headers to each request:

odp_header_key_1: header_value_1
odp_header_key_2: header_value_2

Some analytics platforms use custom headers to determine how to process a request. For details, see the documentation for your analytics platform.

Don't set headers that OMEGAMON Data Connect already sets:

• Accept-Encoding
• Connection
• Content-Encoding
• Content-Length
• Content-Type
• Host
• Transfer-Encoding
• odp-proto
• User-Agent

batching

Optional. When batching is enabled, each request can contain multiple records. Batching offers higher throughput than the default single-record-per-request behavior.

enabled

Whether to enable batching:

true

Enable batching.

Each request contains up to batch-size records, according to the following rules:

• If the number of records in the queue is at least batch-size, then the output sends a request containing batch-size records.
• If there are fewer than batch-size records in the queue, then the output waits for up to the duration specified by linger. If the number of records in the queue reaches batch-size within that duration, then the output sends a request containing batch-size records. Otherwise, if there are still fewer than batch-size records in the queue at the end of that duration, then the output sends a request containing however many records are in the queue.

Each record in the request body is represented by a length-prefixed JSON object. Each JSON object is prefixed by a 4-byte binary-encoded integer, in network byte order, that states the length of the JSON object.

Tip: Before enabling batching, check whether the destination analytics platform supports HTTP POST request bodies that contain a stream of length-prefixed JSON objects, with the length stated in the specific format described here.

Requests contain the header odp-proto: batch.

false

Disable batching.

Each request contains only a single record.

The request body consists of a single JSON object representing a single record. By contrast with the request body format when batching is enabled, this single JSON object has no length prefix.

Requests contain the header odp-proto: single.

This key is optional. Default: false (single record per request).

batch-size

Optional; applies only if batching is enabled. Maximum number of records to send in a request. Default: 1000.

Tip: Before setting a large batch-size, check whether the destination HTTP server will accept the corresponding large requests. Request size is determined by the combination of the number of records in the request and the record lengths. Record lengths can vary depending on factors such as the record types that you have selected to send: different record types involve different numbers of fields and different field lengths. HTTP servers typically have a configuration parameter that sets a maximum request size limit, and reject requests that exceed that limit. For details, see the HTTP server configuration parameter documentation for your destination analytics platform.

linger

Optional; applies only if batching is enabled. Duration to wait if the number of records in the queue is less than batch-size.

Allowed values: a positive integer optionally followed by any of these units:

ns

Nanoseconds

us

Microseconds

ms

Milliseconds

s

Seconds

m

Minutes

h

Hours

d

Days

For example, the value 5m means a duration of 5 minutes.

If the unit is omitted, seconds are used. For example, 2 and 2s are equivalent.

Default: 250ms.

filter

Optional filter to restrict what data to send.

This output-level filter applies only to this endpoint, replacing any global-level filter (connect.filter).

Tip: You can specify an output-level filter for each endpoint (connect.output.http.endpoints.<endpoint_name>.filter) and a global-level filter that applies to all JSON-format outputs (connect.filter). However, you cannot specify a filter that applies only to all HTTP outputs; there is no connect.output.http.filter.