Adding an Amazon AWS CloudTrail log source on the QRadar Console using an SQS queue

If you want to collect AWS CloudTrail logs from multiple accounts or regions in an Amazon S3 bucket, add a log source on the QRadar Console so that Amazon AWS CloudTrail can communicate with QRadar by using the Amazon AWS S3 REST API protocol and a Simple Queue Service (SQS) queue.

Procedure

  1. Use the following table to set the parameters for an Amazon AWS CloudTrail log source that uses the Amazon AWS S3 REST API protocol and an SQS queue.
    Table 1. Amazon AWS S3 REST API protocol log source parameters
    Parameter Description
    Log Source Type Amazon AWS CloudTrail
    Protocol Configuration Amazon AWS S3 REST API
    Log Source Identifier

    Type a unique name for the log source.

    The Log Source Identifier can be any valid value and does not need to reference a specific server. The Log Source Identifier can be the same value as the Log Source Name. If you have more than one Amazon AWS CloudTrail log source that is configured, you might want to identify the first log source as awscloudtrail1, the second log source as awscloudtrail2, and the third log source as awscloudtrail3.
    Authentication Method
    Access Key ID / Secret Key
    Standard authentication that can be used from anywhere.
    Assume IAM Role
    Authenticate with keys and then temporarily assume a role for access. This option is available only when you select SQS Event Notifications for the S3 Collection Method. The supported S3 Collection Method is Use a Specific Prefix.
    EC2 Instance IAM Role
    If your managed host is running on an AWS EC2 instance, choosing this option uses the IAM Role from the instance metadata that is assigned to the instance for authentication; no keys are required. This method works only for managed hosts that are running within an AWS EC2 container.
    Access Key ID

    If you selected Access Key ID / Secret Key for the Authentication Method, the Access Key ID parameter is displayed.

    The Access Key ID that was generated when you configured the security credentials for your AWS user account. This value is also the Access Key ID that is used to access the AWS S3 bucket.

    Secret Key

    If you selected Access Key ID / Secret Key for the Authentication Method, the Secret Key ID parameter is displayed.

    The Secret Key that was generated when you configured the security credentials for your AWS user account. This value is also the Secret Key ID that is used to access the AWS S3 bucket.
    Event Format Select AWS Cloud Trail JSON. The log source retrieves JSON formatted events.
    S3 Collection Method Select SQS Event Notifications.
    SQS Queue URL

    Enter the full URL, starting with https://, of the SQS queue that is set up to receive notifications for ObjectCreate events from S3.
    Region Name The region that the SQS Queue or the S3 Bucket is in.

    Example: us-east-1, eu-west-1, ap-northeast-3
    Use as a Gateway Log Source Select this option for the collected events to flow through the QRadar Traffic Analysis engine and for QRadar to automatically detect one or more log sources.
    Log Source Identifier Pattern

    This option is available when you set Use as a Gateway Log Source is set to yes.

    Use this option if you want to define a custom Log Source Identifier for events being processed. This field accepts key value pairs to define the custom Log Source Identifier, where the key is the Identifier Format String, and the value is the associated regex pattern. You can define multiple key value pairs by entering a pattern on a new line. When multiple patterns are used, they are evaluated in order until a match is found and a custom Log Source Identifier can be returned.
    Show Advanced Options Select this option if you want to customize the event data.
    File Pattern

    This option is available when you set Show Advanced Options to Yes.

    Type a regex for the file pattern that matches the files that you want to pull; for example, .*?\.json\.gz
    Local Directory

    This option is available when you set Show Advanced Options to Yes.

    The local directory on the Target Event Collector. The directory must exist before the AWS S3 REST API PROTOCOL attempts to retrieve events.
    S3 Endpoint URL

    This option is available when you set Show Advanced Options to Yes.

    The endpoint URL that is used to query the AWS REST API.

    If your endpoint URL is different from the default, type your endpoint URL. The default is http://s3.amazonaws.com
    Use S3 Path-Style Access

    Forces S3 requests to use path-style access.

    This method is deprecated by AWS. However, it might be required when you use other S3 compatible APIs. For example, the https://s3.region.amazonaws.com/bucket-name/key-name path-style is automatically used when a bucket name contains a period (.). Therefore, this option is not required, but can be used.
    Use Proxy

    If QRadar accesses the Amazon Web Service by using a proxy, enable Use Proxy.

    If the proxy requires authentication, configure the Proxy Server, Proxy Port, Proxy Username, and Proxy Password fields.

    If the proxy does not require authentication, configure the Proxy Server and Proxy Port fields.
    Recurrence How often a poll is made to scan for new data.

    If you are using the SQS event collection method, SQS Event Notifications can have a minimum value of 10 (seconds). Because SQS Queue polling can occur more often, a lower value can be used.

    If you are using the Directory Prefix event collection method, Use a Specific Prefix has a minimum value of 60 (seconds) or 1M. Because every listBucket request to an AWS S3 bucket incurs a cost to the account that owns the bucket, a smaller recurrence value increases the cost.

    Type a time interval to determine how frequently the poll is made for new data. The time interval can include values in hours (H), minutes (M), or days (D). For example, 2H = 2 hours, 15M = 15 minutes, 30 = seconds.
    EPS Throttle

    The maximum number of events per second that QRadar ingests.

    If your data source exceeds the EPS throttle, data collection is delayed. Data is still collected and then it is ingested when the data source stops exceeding the EPS throttle.

    The default is 5000.
  2. To verify that QRadar is configured correctly, review the following table to see an example of a parsed event message.
    Table 2. Amazon AWS CloudTrail sample message supported by Amazon AWS CloudTrail.
    Event name Low-level category Sample log message
    Console Login General Audit Event 
    {"eventVersion":"1.02",
"userIdentity":{"type":"IAMUser",
"principalId":"XXXXXXXXXXXXXXXXXXXXX",
"arn":"arn:aws:iam::<Account_number>:user/
xx.xxccountId":"<Account_number>","userName":
"<Username>"},"eventTime":
"2016-05-04T14:10:58Z","eventSource":
"f.amazonaws.com","eventName":
"ConsoleLogin","awsRegion":
"us-east-1","sourceIPAddress":
"<Source_IP_address> Agent":"Mozilla/5.0
 (Windows NT 6.1; Win64; x64)
 AppleWebKit/537.36 (KHTML, like Gecko)
 Chrome/50.0.1.1 Safari/537.36",
"requestParameters":null,
"responseElements":
{"ConsoleLogin":"Success"},
"additionalEventData":
{"LoginTo":"www.webpage.com",
"MobileVersion":"No","MFAUsed":"No"},
"eventID":"xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
"eventType":"AwsConsoleSignIn",
"recipientAccountId":"<Account_ID>"}