How to use IBM App Connect with IBM Cloud Object Storage S3

IBM Cloud Object Storage S3 is ideal for holding large amounts of colder production data, such as backups and archives, and very large individual files, such as video files, image files, and genomic data. IBM Cloud Object Storage S3 is a reliable, durable, and resilient object storage.
Availability:
  • App Connect Enterprise as a Service connector
  • A local connector in a Designer instance of IBM App Connect in containers (Continuous Delivery release)Local connector in containers (Continuous Delivery release)
  • A local connector in a Designer instance of IBM App Connect in containers (Long Term Support)Local connector in containers (Long Term Support release)
  • A local connector in a Designer instance of IBM App Connect in containers (Support Cycle 2)Local connector in containers (Long Term Support Cycle-2 release)
The following information describes how to use IBM® App Connect to connect IBM Cloud Object Storage S3 to your other applications.

Supported product and API versions

To find out which product and API versions this connector supports, see Detailed System Requirements on the IBM Support page.

Connecting to IBM Cloud Object Storage

Depending on the type of authentication that your IBM Cloud Object Storage S3 service instance use, first you need to select your preferred authorization method. For considerations regarding your choice of authorization method, see What should I consider first?.

  • If your service instance uses IBM Cloud Identity and Access Management (IAM) authentication, select Provide credentials for App Connect to use (BASIC IAM) and provide the following connection details:
    • Endpoint URL: The Cloud Object Storage service endpoint URL (public only) for your location or region
    • API key: The API key for the Cloud Object Storage service ID
    • Resource instance ID: The unique identifier for the Cloud Object Storage instance
  • If your service instance uses HMAC credentials for authentication, select Provide credentials for App Connect to use (BASIC) and provide the following connection details:
    • Endpoint URL: The Cloud Object Storage service endpoint URL (public only) for your location or region
    • Secret access key: The secret access key of the instance
    • Access key ID: The access key ID of the instance
    • Region: The region of the instance
    • API key: Specify the API key of the instance if the service instance uses the Identity and Access Management (IAM) authentication
    • Resource instance ID: Specify the resource instance ID of the instance if the service instance uses the Identity and Access Management (IAM) authentication
Tip: For a connection to IBM Cloud Object Storage S3, you can only access buckets specific to the location (or region) of the endpoint URL that you have specified. For example, if you connect to the us-geo location (such as the Endpoint URL s3.us.cloud-object-storage.appdomain.cloud), then you can access buckets that are listed with the same location, us-geo, on the Buckets page of your IBM Cloud Object Storage instance. If you want App Connect to access buckets from more than one location, you can create a separate connection for each location.
  • Figure 1. Example of IBM Cloud Object Storage connection details
    Example of IBM Cloud Object Storage connection details

You can find the connection values on the Endpoint and Service credentials pages for the service instance in IBM Cloud:

  1. Log in to IBM Cloud.
  2. From the IBM Cloud Dashboard, click the Cloud Object Storage service instance that you want to work with.
  3. To view the endpoint URLs, click Endpoint in the left pane and select your preferred location or region.
    • If your service instance uses IAM authentication, copy and paste your preferred public endpoint (for example, s3.us.cloud-object-storage.appdomain.cloud) into the App Connect Endpoint URL field.
    • If your service instance supports legacy HMAC authentication, copy and paste your preferred public endpoint (for example, s3.us.cloud-object-storage.appdomain.cloud) into the App Connect Endpoint URL field. Then, copy and paste your preferred location or region (for example, us-geo) into the App Connect Region field.
      Figure 2. IBM Cloud Object Storage service endpoints page
      IBM Cloud Object Storage service endpoints page

      (Click image to view full size.)

  4. To view the service credentials, click Service credentials in the left pane, and then click the drop-down arrow to view credentials. (If you want to define new credentials to use, click New credential.)
    • If your service instance uses IAM authentication, copy the apikey value and paste it into the App Connect API key field. Then, copy the resource_instance_id value and paste it into the App Connect Resource instance ID field.
      Figure 3. IBM Cloud Object Storage credentials page
      IBM Cloud Object Storage credentials page

      (Click image to view full size.)

    • If your service instance supports legacy HMAC authentication, copy the cos_hmac_keys/secret_access_key value and paste it into the App Connect Secret access key field. Then, copy the cos_hmac_keys/access_key_id value and paste it into the App Connect Access key ID field.
      Note: To get the Secret access key and Access key ID values, you need to have a credential created with the option to generate HMAC credentials. (When adding a new credential, specify {"HMAC":true} in the Add Inline Configuration Parameters (Optional) field.

      For more information about creating and managing service credentials, see 'Service credentials' in IBM Cloud Docs / Cloud Object Storage.

      Figure 4. IBM Cloud Object Storage service credentials tab
      IBM Cloud Object Storage service credentials tab

      (Click image to view full size.)

To connect to a IBM Cloud Object Storage S3 endpoint from the App Connect Designer Catalog page for the first time, expand IBM Cloud Object Storage S3, then click Connect. For more information, see Managing accounts.

Tip:

Before you use the account that is created in App Connect in a flow, rename the account to something meaningful that helps you to identify it. To rename the account on the Catalog page, select the account, open its options menu (⋮), then click Rename Account.

What should I consider first?

Before you use App Connect Designer with IBM Cloud Object Storage, take note of the following considerations:

  • Claim check is supported with IBM Cloud Object Storage S3 accounts created in a Cloud environment. See the following table for claim check limits for each authorization method.
    Table 1. IAM and BASIC claim check limits
    Authorization methods File download (MB) File upload (MB)
    IAM 50 50
    BASIC 50 10
    Restriction: Claim check is not supported for IBM Cloud Object Storage S3 accounts in a container environment.
  • For the following ACL actions, the IAM resource access policy for your IBM Cloud Object Storage instance needs to have the "Manager" role:
    Bucket:
    • Create custom ACL for bucket
    • Create standard ACL for bucket
    Object:
    • Create custom ACL for object
    • Create standard ACL for object
    • Retrieve ACLs for objects
    The IAM resource access policy is defined for the service credentials used to connect to the service instance. You can check and configure the access policy at https://cloud.ibm.com/iam/serviceids.
      1. In the Service IDs list, click the name for the IAM API key of your service credentials. You can double-check the row by comparing the description to the "iam_api_key_description" value of your service credentials.

        This displays the Service ID page for the service credentials.

      2. On the Service ID page, select the Access policies tab. The Role column should include Manager.

        To add the Manager role, click the existing role to edit the access policy for the service credentials, then select the Manager checkbox, and then click Save. The Service ID page is shown with the Role column now including the Manager role.

      The Service ID page, displaying the roles assigned for a service credential

      (Click image to view full size.)

  • (General consideration) You can see lists of the trigger events and actions that are available on the Catalog page of the App Connect Designer.

    For some applications, the events and actions in the catalog depend on the environment and whether the connector supports configurable events and dynamic discovery of actions. If the application supports configurable events, you see a Show more configurable events link under the events list. If the application supports dynamic discovery of actions, you see a Show more link under the actions list.

  • (General consideration) If you are using multiple accounts for an application, the set of fields that is displayed when you select an action for that application can vary for different accounts. In the flow editor, some applications always provide a curated set of static fields for an action. Other applications use dynamic discovery to retrieve the set of fields that are configured on the instance that you are connected to. For example, if you have two accounts for two instances of an application, the first account might use settings that are ready for immediate use. However, the second account might be configured with extra custom fields.

Events and actions

IBM Cloud Object Storage S3 events

These events are for changes in this application that trigger a flow to start completing the actions in the flow.

Note: Events are not available for changes in this application. You can trigger a flow in other ways, such as at a scheduled interval or at specific dates and times.

IBM Cloud Object Storage S3 actions

Your flow completes these actions on this application.

Bucket
Create bucket
Retrieve all buckets
Retrieve buckets
Create standard ACL for bucket
Create custom ACL for bucket
CORS
Create CORS configuration for bucket
Retrieve CORS configuration for buckets
Delete CORS configuration for bucket
Object
Create object
Retrieve all objects
Retrieve objects
Download object
Create standard ACL for object
Create custom ACL for object
Update object
Delete object
Retrieve ACLs for objects
Retrieve objects by marker
Search objects
Retrieve object by prefix and delimiter

Examples

Dashboard tile for a template that uses IBM Cloud Object Storage S3

Use templates to quickly create flows for IBM Cloud Object Storage S3

Learn how to use App Connect templates to quickly create flows that complete actions on IBM Cloud Object Storage S3. For example, open the Templates gallery, and then search for IBM Cloud Object Storage S3.

Dashboard tile for a template that uses IBM Cloud Object Storage S3