Creating the SQS queue that is used to receive ObjectCreated notifications

You must create an SQS queue and configure S3 ObjectCreated notifications in the AWS Management Console when using the Amazon AWS S3 REST API connector.

Before you begin

You must complete Finding the S3 Bucket that contains the data that you want to collect. The SQS Queue must be in the same region as the AWS S3 bucket that the queue is collecting from.

Procedure

  1. Log in to the AWS Management Console as an administrator.
  2. Click Services, and then go to the Simple Queue Service Management Console.
  3. In the upper right of the window, change the region to where the bucket is located. You noted this value when you completed the Finding the S3 Bucket that contains the data that you want to collect procedure.
  4. Select Create New Queue, and then type a value for the Queue Name.
  5. Click Standard Queue, select Configure Queue, and then change the default values for the following Queue Attributes.
    • Default Visibility Timeout - 60 seconds (You can use a lower value. In the case of load balanced collection, duplicate events might occur with values of less than 30 seconds. This value can't be 0.)
    • Message Retention Period - 14 days (You can use a lower value. In the event of an extended collection, data might be lost.)

    Use the default value for the remaining Queue Attributes.

    More options such as Redrive Policy or SSE can be used depending on the requirements for your AWS environment. These values should not affect the data collection.

    This image shows a new Standard SQS Queue that has been created and Queue Attributes.

    Picture © 2019 Amazon.com Inc. or its subsidiaries. All Rights Reserved.

  6. Select Create Queue.

What to do next

Setting up SQS queue permissions