IBM Cloud Object Storage protocol configuration options

The IBM Cloud Object Storage protocol for IBM QRadar is an outbound or active protocol that collects logs that are contained in objects from IBM Cloud Object Storage buckets.

Important: Before you configure the IBM Cloud Object Storage protocol, configure user access roles and service credentials to access the IBM Cloud Object Storage buckets.

You must have either the Reader, Writer, or Manager role to access the buckets. For more information about user access roles and permissions, see Bucket permissions (https://cloud.ibm.com/docs/cloud-object-storage?topic=cloud-object-storage-iam-bucket-permissions).

You must create service credentials that include hash-based message authentication code (HMAC) credentials. For more information about service credentials, see Using HMAC credentials (https://cloud.ibm.com/docs/cloud-object-storage?topic=cloud-object-storage-uhc-hmac-credentials-main).

Table 1. IBM Cloud Object Storage protocol common log source parameters
Parameter	Description
Protocol Configuration	IBM Cloud Object Storage
Log Source Identifier	Type a unique name for the log source. The log source identifier does not need to reference a specific server, and it can be the same value as the Log Source Name.
HMAC Access Key ID	The Access Key ID that was generated when you configured the service credentials.
HMAC Secret Access Key	The Secret Access Key that was generated when you configured the service credentials.
Endpoint	The public endpoint that is stated in the bucket configuration page.
Bucket Name	The name of the bucket that logs are stored in.
Prefix	The prefix filter value to limit collecting objects or file keys that begin with the prefix. To pull all files from the bucket, use a forward slash (/). Important: Changing the Prefix value clears the persisted file marker. All files that match the new prefix are downloaded in the next pull. If the Prefix file path is used to specify folders, you must not begin the file path with a forward slash. For example, use folder1/folder2 instead.
Event Format	The following event formats are supported: LINEBYLINE Raw log files that contain one record per line. You can use either .gz, .gzip, or .zip files for compression. W3C Files that contain generic W3C formatting data to output name-value-pair events (.gz files only).
Use As A Gateway Log Source	If you do not want to define a custom log source identifier for events, clear the checkbox. If you don't select Use As A Gateway Log Source and you don't configure the Log Source Identifier Pattern, QRadar receives events as unknown generic log sources.
Log Source Identifier Pattern	If you select Use As A Gateway Log Source, you can define a custom log source identifier. Use this option for events that are being processed and for log sources that are automatically discovered. If you don't configure the Log Source Identifier Pattern, QRadar receives events as unknown generic log sources. Use key-value pairs to define the custom log source identifier. The key is the identifier format string, which is the resulting source or origin value. The value is the associated regex pattern that is used to evaluate the current payload. This value also supports capture groups that can be used to further customize the key. Define multiple key-value pairs by typing each pattern on a new line. Multiple patterns are evaluated in the order that they are listed. When a match is found, a custom log source identifier is displayed. The following examples show multiple key-value pair functions: `Patterns VPC=\sREJECT\sFAILURE $1=\s(REJECT)\sOK VPC-$1-$2=\s(ACCEPT)\s(OK) Events {LogStreamName: LogStreamTest,Timestamp: 0,Message: ACCEPT OK,IngestionTime: 0,EventId: 0} Resulting custom log source identifier VPC-ACCEPT-OK`
Show Advanced Options	To configure the advanced options for event collection, set this option to On.
File Pattern	Type a regex for the file pattern that matches the files that you want to pull; for example, `.*?\.json\.gz`. This option is available when you set Show Advanced Options to On.
Local Directory	The local directory on the Target Event Collector. The directory must exist before the protocol attempts to retrieve events. This option is available when you set Show Advanced Options to on.
Use Proxy	If QRadar accesses the IBM Cloud Object Storage by using a proxy, enable Use Proxy. If the proxy requires authentication, configure the Proxy Server, Proxy Port, Proxy Username, and Proxy Password parameters. If the proxy does not require authentication, leave the Proxy Username and Proxy Password fields blank.
Recurrence	Type a time interval to determine how frequently the protocol polls for new data. The time interval can include values in hours (H), minutes (M), or days (D). For example, 2H = 2 hours, 15M = 15 minutes, 30 = seconds. The minimum value is 60 (seconds) or 1M.
EPS Throttle	The maximum number of events per second that QRadar ingests. If your data source exceeds the EPS throttle, data collection is delayed. Data is still collected and then it is ingested when the data source stops exceeding the EPS throttle. The default is 5000.