Connecting to an Amazon CloudWatch data source

Connect the Amazon CloudWatch data source to IBM Security QRadar® Suite Software to enable your applications and dashboards to collect and analyze Amazon CloudWatch security data. Universal Data Insights connectors enable federated search across your security products.

Before you begin

Collaborate with an AWS administrator to obtain a user account with access to query the CloudWatch data source.

If you have a firewall between your cluster and the data source target, use the IBM® Security Edge Gateway to host the containers. The Edge Gateway must be V1.6 or later. For more information, see Edge Gateway.

About this task

With Amazon CloudWatch, you can set up centralized access to the logs from all of your Amazon systems, applications, and Amazon Web Services (AWS) in a single service.

IBM Security QRadar Suite Software currently supports the data source connection for logs of Amazon GuardDuty and VPC Flow.

Structured Threat Information eXpression (STIX) is a language and serialization format that organizations use to exchange cyberthreat intelligence. The connector uses STIX patterning to query Amazon CloudWatch data and returns results as STIX objects. For more information about how the Amazon CloudWatch data schema maps to STIX, see Amazon CloudWatch stix-shifter repository (https://github.com/opencybersecurityalliance/stix-shifter/tree/develop/stix_shifter_modules/aws_cloud_watch_logs).

Procedure

  1. Log in to IBM Security QRadar Suite Software.
  2. From the menu, click Connections > Data sources.
  3. On the Integration data sources page, on the Amazon CloudWatch Logs tile, click Set up a connection.
  4. On the Connection services page, select the Federated searches service tile, and then click Enable.
    The available connection services include:
    1. Connected assets & risk
    2. Federated searches
    Note: If there are multiple data sources to connect, select the connector from the Sources list, and then click Enable.
  5. Click Next.
  6. On the Connection details page, configure the following parameters.
    1. Configure the connection to the data source.
      Table 1. Connection parameters
      Parameter Description
      Data source name Enter a unique name to identify the data source connection. You can create multiple connections to a data source, so it is useful to clearly set them apart by name.

      Only alphanumeric characters and the following special characters are allowed: - . _

      Data source description Enter a description to indicate the purpose of the data source connection. You can create multiple connections to a data source, so it is useful to clearly indicate the purpose of each connection by description.

      Only alphanumeric characters and the following special characters are allowed: - . _

      Edge gateway If you have a firewall between your cluster and the data source target, use the Edge Gateway to host the containers. In the Edge gateway field, specify an Edge Gateway to host the connector.

      It can take up to five minutes for the status of newly deployed data source connections on the Edge Gateway to show as being connected.

      Region Enter the region for your CloudWatch data source. Select your region code from the Region column of the Service Endpoints table in the AWS General Reference guide (https://docs.aws.amazon.com/general/latest/gr/cwl_region.html).
      Log group names (Optional) In the Log group names field, specify the log group names of the CloudWatch logs that you want to connect to. If the log group names are not specified, all the available log groups are connected.
      The value of Log group names must use a specific JSON format:
      {"<service_type>": "<service_log_group_name>"}
      For example:
      {"vpcflow": "vpcflow_log_group_name"}
      To get logs from multiple service types, the service types and their associated log group names can be specified in the JSON separated by a comma. For example:
      {"vpcflow": "vpcflow_log_group_name", "guardduty": "guardduty_log_group_name"}
    2. Set the query parameters to control the behavior of the federated search query on the data source.
      Table 2. Query parameters
      Query parameter Description
      Concurrent search limit Enter the number of simultaneous connections that can be made to the data source. The default limit for the number of connections is 4. The value must not be less than 1 and must not be greater than 100.
      Query search timeout limit Enter the time limit in minutes for how long the query is run on the data source. The default time limit is 30. When the value is set to zero, no timeout occurs. The value must not be less than 1 and must not be greater than 120.
      Result size limit Enter the maximum number of entries or objects that are returned by search query. The default result size limit is 10,000. The value must not be less than 1 and must not be greater than 500,000.
      Query time range Enter the time range in minutes for the search, represented as the last X minutes. The default is 5 minutes. The value must not be less than 1 and must not be greater than 10,000.
      Custom mapping (Optional) If you need to customize the STIX attributes mapping, click Customize attribute mapping and edit the JSON blob to map new or existing properties to their associated target data source fields.
      Important: If you increase the Concurrent search limit and the Result size limit, a greater amount of data can be sent to the data source, which increases the strain on the data source. Increasing the query time range also increases the amount of data.
    3. If CloudWatch is configured with a self-signed Security Sockets Layer (SSL) certificate, add a connection certificate.

      To confirm that you have a self-signed certificate, you can search the web for 'ssl decode', then copy and paste your certification into a Certificate Decoder. If the value in the Common Name field is local, for example: yourlocalhost.yourlocaldomain, it indicates a self-signed certificate.

      If the value of your hostname or IP address does not match the common name value in your certificate, add the hostname to the server that hosts the Edge Gateway.
      Table 3. Adding your hostname to the Edge Gateway server
      If you are running the Edge client If you are not running the Edge client
      Run the following command:
      manageAppHost dns --set --ip <ip_address> --hostname <hostname1> --hostname <hostname2>
      Tip: The <ip_address> is the address of the server that you want to access, and <hostname> is the name of the server. If a server has multiple names, you can add each name by using --hostname.

      You can use --show to see the defined hostnames and --clear to remove a hostname previously set with the manageAppHost command.

      Add the hostname to the server that hosts the Edge Gateway by adding entries to Pod /etc/hosts with HostAliases. To edit the Universal Data Insights worker deployment, run the following command:
      oc edit deployment udi-udiworker
      Add the following information to the host deployment HostAliases, in the spec.template.spec section:
      hostAliases:
         - hostnames:
          - qradar.iseccs.local
          ip: <1.2.3.4>

      For more information, see Adding entries to Pod /etc/hosts with HostAliases (https://kubernetes.io/docs/tasks/network/customize-hosts-file-for-pods/).

    4. Copy the certificate details and paste it in the space that is provided.
    5. Click Next.
  7. On the Connection configurations page, configure identity and access.
    1. Click Add a configuration.
    2. In the Configuration details window, configure the following parameters.
      Table 4. Configuration parameters
      Parameter Description
      Configuration Name Enter a unique name to describe the access configuration and distinguish it from the other access configurations for this data source connection that you might set up. Only alphanumeric characters and the following special characters are allowed: - . _
      Configuration Description Enter a unique description to describe the access configuration and distinguish it from the other access configurations for this data source connection that you might set up. Only alphanumeric characters and the following special characters are allowed: - . _
      AWS access key ID Establish AWS authentication to enable access to the AWS search API.

      To establish an AWS key-based authentication, enter values for the AWS Access key id and AWS secret access key parameters.

      To establish an AWS role-based authentication, enter values for the AWS Access key id, AWS secret access key, and AWS IAM Role parameters. The AWS IAM Role parameter is the ARN (Amazon Resource Name) of the selected IAM Role.

      AWS secret access key
      AWS IAM role
      Important: You must establish AWS authentication to enable access to the AWS search API. For more information about AWS authentication, see Configuring AWS authentication.
    3. To save your configuration and establish the connection, click Save.
  8. Click Next.
  9. To assign user access, on the User access control page, select one or more data source configurations from the Access list, and then click Finish.
  10. To manage your active connections, complete the following steps:
    1. On the Integration data sources page, on the tile of the relevant data source, click Manage <x> of <x> active connections.
    2. On the Connection status page, on the tile of the relevant data source, you can edit, refresh, or delete your data source connection.

Results

After you connect a data source, it might take up to 30 seconds to retrieve the data. Before the full data set is returned, the data source might display as unavailable. After the data is returned, the data source shows as being connected, and a polling mechanism occurs to validate the connection status. The connection status is valid for 60 seconds after every poll.

You can add other connection configurations for this data source that have different users and different data access permissions.

What to do next

Test the connection by searching for an IP address in IBM Security Data Explorer that matches an asset data source. In Data Explorer, click an IP address to view its associated assets and risk.

To use Data Explorer, you must have data sources that are connected so that the application can run queries and retrieve results across a unified set of data sources. The search results vary depending on the data that is contained in your configured data sources. For more information about how to build a query in Data Explorer, see Build a query.