How to use IBM® App Connect with Amazon S3

Amazon Simple Storage Service is a simple storage service that provides object storage through a web service interface. It uses the scalable storage infrastructure of Amazon.com and enables you to store objects, download and use data with other AWS services, and build applications that call for internet storage.

Availability:
  • App Connect connector
  • A local connector in a Designer instance of IBM App Connect in containers (Continuous Delivery release)Local connector in containers (Continuous Delivery release)
  • A local connector in a Designer instance of IBM App Connect in containers (Support Cycle 2)Local connector in containers (Long Term Support Cycle-2 release)

Supported product and API versions

To find out which product and API versions this connector supports, see Detailed System Requirements on the IBM Support page.

Connecting to Amazon S3

Complete the connection fields that you see in the App Connect Designer Connect > Applications and APIs page (previously the Catalog page) or flow editor. If necessary, work with your Amazon S3 administrator to obtain these values.

Amazon S3Authorization type and connection fields:

BASIC
Secret access key: The secret access key for your Amazon S3 account, as generated in the Security Credentials page in the AWS Management Console.
Access key ID: The access key ID for your Amazon S3 account, as generated in the Security Credentials page in the AWS Management Console.
Role ARN: The Amazon Resource Name (ARN) that specifies an IAM role in AWS.
Region: The region of your Amazon S3 instance, for example, us-east-1. You can find the value for the Region parameter at the end of the URL when you are logged in to the AWS Management Console (for example, https://us-east-2.console.aws.amazon.com/console/home?region=us-east-2#).
Tip: For more information, see AWS service endpoints on the AWS documentation page.
AWS PrivateLink or VPC endpoint: The endpoint type that connects your VPC to AWS services or third-party services without using the public internet. Use AWS PrivateLink (App Connect Enterprise as a Service) for private connectivity over the AWS network, or use a VPC endpoint (App Connect Enterprise Toolkit) to enable secure, private communication with supported AWS services. If you are using a VPC endpoint, the URL must be in the format bucket.<Amazon S3 VPC interface endpoint> (for example, bucket.vpce-0a5422b2fc0329a4e-83pq101q.s3.us-east-1.vpce.amazonaws.com).
AWS PrivateLink certificate or TLS Certificate: Enter the AWS PrivateLink certificate (App Connect Enterprise as a Service) or a TLS certificate (App Connect Enterprise Toolkit) for connecting to your Amazon S3 instance in PEM format.
Bucket name: Specify the bucket name in your Amazon S3 account, if you only have access to specific buckets in your Amazon S3 account.
BASIC OIDC
Region: The region of your Amazon S3 instance, for example, us-east-1. You can find the value for the Region parameter at the end of the URL when you are logged in to the AWS Management Console (for example, https://us-east-2.console.aws.amazon.com/console/home?region=us-east-2#).
Tip: For more information, see AWS service endpoints on the AWS documentation page.
AWS PrivateLink or VPC endpoint: The endpoint type that connects your VPC to AWS services or third-party services without using the public internet. Use AWS PrivateLink (App Connect Enterprise as a Service) for private connectivity over the AWS network, or use a VPC endpoint (App Connect Enterprise Toolkit) to enable secure, private communication with supported AWS services. If you are using a VPC endpoint, the URL must be in the format bucket.<Amazon S3 VPC interface endpoint> (for example, bucket.vpce-0a5422b2fc0329a4e-83pq101q.s3.us-east-1.vpce.amazonaws.com).
AWS PrivateLink certificate or TLS Certificate: Enter the AWS PrivateLink certificate (App Connect Enterprise as a Service) or a TLS certificate (App Connect Enterprise Toolkit) for connecting to your Amazon S3 instance in PEM format.
Bucket name: Specify the bucket name in your Amazon S3 account, if you only have access to specific buckets in your Amazon S3 account.
Client ID: Specify the unique identifier assigned to an application within an OpenID Connect (OIDC) system.
Client secret: Specify the client secret that is used to authenticate the client application.
ID token: The security token in OpenID Connect (OIDC) that contains claims about the authentication of a user, such as their identity and session validity, typically represented as a JSON Web Token (JWT).
Refresh token: The refresh token that is generated from the application client ID and client secret.
Role ARN: The Amazon Resource Name (ARN) of the IAM role that defines the permissions that are applied when the role is assumed.
OIDC server URL: Specify the URL of the OpenID Connect (OIDC) server or identity provider that handles authentication and provides tokens for clients.
OIDC WEB
Region: The region of your Amazon S3 instance, for example, us-east-1. You can find the value for the Region parameter at the end of the URL when you are logged in to the AWS Management Console (for example, https://us-east-2.console.aws.amazon.com/console/home?region=us-east-2#).
Tip: For more information, see AWS service endpoints on the AWS documentation page.
AWS PrivateLink or VPC endpoint: The endpoint type that connects your VPC to AWS services or third-party services without using the public internet. Use AWS PrivateLink (App Connect Enterprise as a Service) for private connectivity over the AWS network, or use a VPC endpoint (App Connect Enterprise Toolkit) to enable secure, private communication with supported AWS services. If you are using a VPC endpoint, the URL must be in the format bucket.<Amazon S3 VPC interface endpoint> (for example, bucket.vpce-0a5422b2fc0329a4e-83pq101q.s3.us-east-1.vpce.amazonaws.com).
AWS PrivateLink certificate or TLS Certificate: Enter the AWS PrivateLink certificate (App Connect Enterprise as a Service) or a TLS certificate (App Connect Enterprise Toolkit) for connecting to your Amazon S3 instance in PEM format.
Bucket name: Specify the bucket name in your Amazon S3 account, if you only have access to specific buckets in your Amazon S3 account.
Client ID: Specify the unique identifier assigned to an application within an OpenID Connect (OIDC) system.
Client secret: Specify the client secret that is used to authenticate the client application.
Role ARN: The Amazon Resource Name (ARN) of the IAM role that defines the permissions that are applied when the role is assumed.
OIDC server URL: Specify the URL of the OpenID Connect (OIDC) server or identity provider that handles authentication and provides tokens for clients.
AWS PKI
Region: The region of your Amazon S3 instance, for example, us-east-1. You can find the value for the Region parameter at the end of the URL when you are logged in to the AWS Management Console (for example, https://us-east-2.console.aws.amazon.com/console/home?region=us-east-2#).
Tip: For more information, see AWS service endpoints on the AWS documentation page.
Bucket name: Specify the bucket name in your Amazon S3 account, if you only have access to specific buckets in your Amazon S3 account.
Client certificate: The X.509 certificate used to authenticate your workload with IAM Roles Anywhere.
Client key password: The password for the encrypted Client private key. Required only if the Client private key is protected by a password.
Client private key: The private key that is associated with the client certificate and used to sign authentication requests.
Profile ARN: The Amazon Resource Name (ARN) of the IAM Roles Anywhere profile that determines the IAM roles that a workload can assume.
Role ARN: The Amazon Resource Name (ARN) of the IAM role that defines the permissions that are applied when the role is assumed.
Trust anchor ARN: The Amazon Resource Name (ARN) of the trust anchor that represents the certificate authority (CA) trusted by IAM Roles Anywhere to validate X.509 client certificates.

To obtain the connection values for Amazon S3 using BASIC, BASIC OIDC, and OIDC WEB authentication types, see Obtaining connection values for Amazon S3 (BASIC, OIDC WEB, and BASIC OIDC). To configure AWS PrivateLink or VPC endpoint, and AWS PrivateLink certificate or TLS Certificate, see Configuring AWS PrivateLink endpoint or VPC endpoint and AWS PrivateLink certificate or TLS Certificate for Amazon S3.

To obtain the connection values for Amazon S3 using AWS PKI authentication type, see Obtaining connection values for Amazon S3 (AWS PKI).

To connect to an Amazon S3 endpoint from the App Connect Designer Applications and APIs page for the first time, expand Amazon S3, then click Connect. For more information, see Managing accounts.

General considerations

Before you use App Connect Designer with Amazon S3, take note of the following considerations:

  • (General consideration) You can see lists of the trigger events and actions that are available on the Applications and APIs page of the App Connect Designer.

    For some applications, the events and actions depend on the environment and whether the connector supports configurable events and dynamic discovery of actions. If the application supports configurable events, you see a Show more configurable events link under the events list. If the application supports dynamic discovery of actions, you see a Show more link under the actions list.

  • (General consideration) If you are using multiple accounts for an application, the set of fields that is displayed when you select an action for that application can vary for different accounts. In the flow editor, some applications always provide a curated set of static fields for an action. Other applications use dynamic discovery to retrieve the set of fields that are configured on the instance that you are connected to. For example, if you have two accounts for two instances of an application, the first account might use settings that are ready for immediate use. However, the second account might be configured with extra custom fields.

Amazon S3 connector limitations

Take note of the following limitations after you are connected to Amazon S3.
  • After you create a bucket, you cannot change its Region or Name.
  • A bucket name must comply with DNS naming rules that are defined by Amazon. For example, a bucket name must not contain periods (.).
  • If you specify an Amazon Region in the Create bucket action that differs from the Region you specified when you created the connection to an Amazon S3 client, the request might fail.

    To create an object in a Region that is different from the Region that you specified when you created the connection, change the Region setting in the AWS SDK. For more information, see Set the AWS Region on the AWS Developer Guide page. After you create the object, reset the Region setting to the default Region before you use other actions. Frequent changes to the Region setting can cause inconsistent behavior.

  • By default, you can create up to 100 buckets in each Amazon account. To increase this limit, submit a service limit increase request.
  • You cannot create a bucket inside another bucket.
  • When you add many new objects encrypted with AWS KMS after you enable cross‑Region replication (CRR), you might receive throttling errors (HTTP 503 “Slow Down”).
  • You can configure up to 1000 metrics configurations for each bucket.
  • Daily storage metrics are not available for filtered subsets of objects.
  • Metric completeness and timeliness are not guaranteed.
  • Generating a .torrent file for an object requires time proportional to the object size.
  • An ACL can contain up to 100 grants.
  • You can specify a grantee by using an email address only in the following Amazon Regions:
    • US East (N. Virginia)
    • US West (N. California)
    • US West (Oregon)
    • Asia Pacific (Singapore)
    • Asia Pacific (Sydney)
    • Asia Pacific (Tokyo)
    • EU (Ireland)
    • South America (São Paulo)
  • You can specify only one canned Access Control List (ACL) in a request.
  • You can configure multiple storage‑class analysis filters for a bucket, up to 1000 filters. Each filter generates a separate analysis.
  • For Reduced Redundancy Storage (RRS), the average annual expected loss is 0.01% of stored objects. If an RRS object is lost, Amazon S3 returns a 405 error when you try to access the object.
  • You might receive HTTP 503 “Slow Down” responses for Amazon S3 PUT or DELETE requests to a versioned bucket that contains objects with millions of versions.
  • Each S3 bucket can have up to 10 tags in its tag set.
  • Each object can have up to 10 tags, and the tag keys must be unique within the object.
  • A tag key can contain up to 128 Unicode characters, and a tag value can contain up to 256 Unicode characters.
  • A lifecycle configuration can contain up to 1000 rules. The ID element uniquely identifies a rule and can contain up to 255 characters.

Events and actions

Amazon S3 events

These events are for changes in this application that trigger a flow to start completing the actions in the flow.

Note: Events are not available for changes in this application. You can trigger a flow in other ways, such as at a scheduled interval or at specific dates and times.

Amazon S3 actions

Your flow completes these actions on this application.

Object Action Description
Buckets Create bucket Creates a bucket belonging to an Amazon S3 account
Retrieve buckets Retrieves details of buckets belonging to an Amazon S3 account
Object ACLs Update object ACL Updates the object ACL configuration
Object tags Delete object tags Deletes object tags
Update object tags Updates the object tags
Objects Create object Create an object in an Amazon S3 bucket using multipart
Delete object Delete an object from an Amazon S3 bucket
Retrieve object metadata Retrieve objects by key from an Amazon S3 bucket
Update or create object Update or create an object in an Amazon S3 bucket

More items are available when you have connected App Connect to Amazon S3.

Examples

Dashboard tile for a template that uses Amazon S3
Dashboard tile for a template that uses Amazon S3
Dashboard tile for a template that uses Amazon S3

Use templates to quickly create flows for Amazon S3

Learn how to use App Connect templates to quickly create flows that complete actions on Amazon S3. For example, open Discover, and then search for Amazon S3.

Learn more

Dashboard tile for a template that uses Amazon S3