Splunk
The Splunk destination writes data to Splunk using the Splunk HTTP Event Collector (HEC). For information about supported versions, see Supported Systems and Versions.
The destination sends HTTP POST requests to the HEC endpoint using the JSON data format. The destination generates one HTTP request for each batch, sending multiple records at a time. Each record must contain the event data and optionally the event metadata in the format required by Splunk.
Before you configure the destination, you must complete several prerequisites including enabling HEC in Splunk and creating an HEC authentication token.
When you configure the Splunk destination, you supply the Splunk API endpoint and the HEC authentication token. You can optionally use an HTTP proxy and configure SSL/TLS properties. You can also use a connection to configure the destination.
You can configure the timeout, request transfer encoding, and authentication type. You can configure the destination to use the Gzip or Snappy compression format to write the data.
You can also configure the destination to log request and response information.
Prerequisites
- Enable HTTP Event Collector (HEC)
- By default, HEC in Splunk is disabled. Enable HEC as described in the Splunk documentation.
- Create an HTTP Event Collector (HEC) token
- To send data to HEC, the Splunk destination must use a token to authenticate to the Splunk server on which HEC is running. Create the HEC token as described in the Splunk documentation.
Required Record Format
Splunk requires that the event data and metadata be correctly formatted in the record. If the record is formatted incorrectly, an error occurs and the destination fails to write to Splunk. When you design a pipeline with the Splunk destination, you must ensure that the record sent to the destination uses the required format.
The record can optionally contain event metadata fields. Splunk includes several pre-defined keys that can be included in the event metadata. Any metadata key-value pairs that are not included in the event are set to values defined for the token on the Splunk server. For a list of the keys that can be included in event metadata, see Event metadata in the Splunk documentation.
{
"time": 1437522387,
"host": "myserver.example.com",
"source": "myapp",
"event": {
"message": "Here is my message",
"severity": "INFO"
}
}
{
"time": 1426279439, // epoch time
"host": "localhost",
"source": "datasource",
"sourcetype": "txt",
"index": "main",
"event": "Here is my event"
}
Logging Request and Response Data
The Splunk destination can log request and response data to the Data Collector log.
When enabling logging, you configure the following properties:
- Verbosity
-
The type of data to include in logged messages:
- Headers_Only - Includes request and response headers.
- Payload_Text - Includes request and response headers as well as any text payloads.
- Payload_Any - Includes request and response headers and the payload, regardless of type.
- Log Level
- The level of messages to include in the Data Collector log. When you select a level, higher level messages are also logged. That is, if you select the Warning log level, then Severe and Warning messages are written to the Data Collector log.
- Max entity size
-
The maximum size of message data to write to the log. Use to limit the volume of data written to the Data Collector log for any single message.
Configuring a Splunk Destination
Configure a Splunk destination to write data to Splunk using the Splunk HTTP Event Collector (HEC).