Configuring cloud storage volumes

Cloud storage products that you can use with IBM Content Manager

For the resource manager storage volume, you can use the following products:

• IBM Cloud Object Storage (ICOS)
• IBM Cloud Object Storage (ICOS) with retention enabled (see Support for IBM Cloud Object Storage retention)
• IBM Cloud Object Storage (ICOS) with object lock (IBM Content Manager V8.7 fix pack 2 and later) (see Support for cloud object storage with object lock)
• Amazon Simple Storage Service (Amazon S3) (see Using Amazon Simple Storage Service)
• Dell EMC Elastic Cloud Storage (Dell EMC ECS) (IBM Content Manager V8.7 fix pack 1 and later)
• Google Cloud Storage (see Using Google Cloud Storage)
• Hitachi Content Platform (HCP)
• Hitachi Content Platform (HCP) for Cloud Scale (HCP for cloud scale) (IBM Content Manager V8.7 fix pack 1 and later)
• Hitachi Content Platform (HCP) for Cloud Scale (HCP for cloud scale) with object lock (IBM Content Manager V8.7 fix pack 2 and later)
• Microsoft Azure Block Blob Storage.

Using Amazon Simple Storage Service

• If the resource manager runs in an Amazon Elastic Compute Cloud (Amazon EC2) instance with Amazon Identity and Access Management (IAM) role attached, you can configure the resource manager to access Amazon S3 by using the IAM role: see Using a cloud storage system. To configure the IAM role for EC2 instances, see the Amazon documentation.
• The Intelligent-Tiering configurations in Amazon S3 and storage classes other than the AWS S3 Standard have not been tested with the resource manager.
• Storage classes such as Glacier Instant Retrieval, and the access tiers which support instant retrieval and comply with the Standard Amazon S3 REST APIs, as long as they are transparent to the resource manager, are supported.
• Storage classes and access tiers which do not support instant retrieval or do not comply with the Standard Amazon S3 REST APIs, are not supported. These include Archive Access tier, Deep Archive Access tier, Glacier Flexible Retrieval storage class, and Glacier Deep Archive storage class, and so on.

Using Google Cloud Storage

The resource manager accesses the Google Cloud Storage by using the Cloud Storage XML API, which is compatible with the Amazon S3 API. To use Google Cloud Storage to store your objects, you must do the following:

1. Set up your default project and bucket.
2. Get the Google Cloud Storage Hash-base Message Authentication Code (HMAC) access ID and secret for your credentials.
3. Make sure that the bucket that you set up for the resource manager is:
   • not enabled with retention
   • not enabled with versioning
   • not enabled with event-based hold.

For more information, see "Simple migration" in the Google Cloud Storage documentation.

Support history

IBM Cloud Object Storage support was added in V8.6.

IBM Cloud Object Storage with retention support was added in V8.6 fix pack 2.

IBM Cloud Object Storage with object lock was added in V8.7 fix pack 2.

Amazon S3 support was added in V8.6 fix pack 1.

Hitachi Content Platform (HCP) support was added in V8.6 fix pack 3.

Microsoft Azure Block Blob Storage, Amazon S3 with Object Lock, and Google Cloud Storage support were added in V8.7.

Hitachi Content Platform (HCP) for Cloud Scale (HCP for cloud scale) and Dell EMC Elastic Cloud Storage support were added in V8.7 fix pack 1.

Hitachi Content Platform (HCP) for Cloud Scale (HCP for cloud scale) with object lock was added in V8.7 fix pack 2.

Using a cloud storage system

To use one of the supported cloud storage systems, you must define a cloud object storage server that connects to it.

Using versioning

See:

Using versioning in V8.7 GA

Using versioning in V8.7 fix pack 1 and later

Using versioning in V8.7 GA

The resource manager does not require versioning to be enabled on the following cloud object storage server buckets or containers:

• IBM Cloud Object Storage
• Amazon S3 buckets without object lock enabled
• Google Cloud Storage
• Hitachi Content Platform (HCP) buckets
• Microsoft Azure Block Blob Storage containers.

If versioning is enabled on these buckets or containers, the objects that are deleted by the resource manager remain on the cloud object storage server with their version ID, and they are no longer managed by the resource manager. To avoid this, ensure that the versioning is not enabled on these types of cloud object storage buckets or containers that you are using with the resource manager.

Versioning is enforced by the following cloud object storage server bucket:

• Amazon S3 buckets with object lock enabled.

When this type of cloud object storage bucket is used by the resource manager, the objects that are deleted by the resource manager are removed permanently by using their version ID.

Using versioning in V8.7 fix pack 1 and later

The resource manager does not require versioning to be enabled on the following cloud object storage server buckets or containers:

If versioning is enabled on the following buckets, the objects that are deleted by the resource manager are removed permanently from the cloud object storage server by using their version ID. To keep the objects with their version ID on the cloud object storage server when they are deleted from resource manager, use the resource manager admin console to set the COS_ALWAYS_USE_NON_VERSIONED_DELETE parameter to TRUE. After they are deleted from resource manager, the objects with their version ID stay on the cloud object storage server; become a non-current version; and are no longer managed by the resource manager.

• IBM Cloud Object Storage
• IBM Cloud Object Storage with object lock enabled
• Amazon S3 buckets with object lock enabled
• Amazon S3 buckets without object lock enabled
• Dell EMC Elastic Cloud Storage with or without object lock enabled
• Google Cloud Storage
• Hitachi Content Platform for Cloud Scale (HCP for cloud scale) without object lock enabled.
• Hitachi Content Platform for Cloud Scale (HCP for cloud scale) with object lock enabled.

If versioning is enabled on the following buckets or containers, the objects that are deleted by the resource manager stay on the cloud object storage server with their version ID. They become a non-current version, and they are no longer managed by the resource manager. To avoid this, ensure that the versioning is not enabled on these types of cloud object storage buckets or containers that you are using with the resource manager.

• Hitachi Content Platform (HCP) buckets in versions that do not support "delete by version".

  "Delete by version" was added to Hitachi Content Platform (HCP) in version 9.4.0.

• Microsoft Azure Block Blob Storage containers.

Defining a Cloud Object Storage server

1. In the cloud storage system, create a bucket (or if you are using Microsoft Azure Block Blob Storage, a container) to use as the resource manager storage volume.

   For more information about how to do this, see the documentation for your cloud storage system.

2. In the Content Manager Enterprise Edition System Administration client, navigate to Content Manager > library server > Resource Managers > resource manager.
3. Define the Cloud Object Storage server: Right-click Server Definitions, and then click New.

   The New Server Definition dialog box appears.

4. Set these properties:

   Name

   Type a name to identify the server, for example: S3Server or AzureServer.

   Server type

   Select Cloud Object Storage.

   Hostname

   The name or IP address of the server that the cloud storage system is on.

   V8.7 GA

   You must use a path-style URL; do not use a virtual hosted-style URL.

   For example, if you are using Amazon S3, use a path-style URL hostname that looks like this:

   s3.<region>.amazonaws.com

   Do not use a virtual hosted-style URL hostname that looks like this:

   <bucketname>.s3.<region>.amazonaws.com

   After you add a volume using path-style URL in the resource manager, do not change the path-style URL hostname in the server definition to a virtual hosted-style URL; if you change it, the existing objects cannot be accessed.

   V8.7 fix pack 1 and later

   You can use a path-style URL or a virtual hosted-style URL. To use a virtual hosted-style URL, select Enable virtual hosted-style requests.

   For example, if you are using Amazon S3, you can use one of the following:

   • a path-style hostname that looks like this:

     s3.<region>.amazonaws.com

   • a virtual hosted-style hostname that looks like this:

     <bucketname>.s3.<region>.amazonaws.com

   If you are using Microsoft Azure Block Blob Storage containers, you must use a path-style URL; do not select Enable virtual hosted-style requests.

   For more information about path-style and virtual hosted-style hostnames, see your cloud storage server's documentation.

   Access key ID

   For Microsoft Azure Block Blob Storage: the account name.

   For other cloud storage servers: the access key id.

   If you want to use Amazon IAM role authentication, leave this field blank.

   Secret access key

   For Microsoft Azure Block Blob Storage: the account key.

   For other cloud storage servers: the secret access key.

   If you want to use Amazon IAM role authentication, leave this field blank.

   Protocol

   Select http or https.

   Important: If you use HTTPS, you must import your S3 provider’s certificate into the WebSphere server that runs the resource manager.

   Port number

   The port number on which the cloud storage server listens.

   The default value for http is 80; the default value for https is 443.

   Path or Region

   The label of this field will show as Region instead of Path since IBM Content Manager V8.7 fix pack 2.

   For Microsoft Azure Block Blob Storage, leave this field blank.

   For other cloud storage servers, enter the region name to be used by the AWS4 authentication system to generate the authentication signature. If the region name is "us-standard", you can leave this field blank.

   Amazon IAM role authentication

   To use Amazon Identity and Access Management (IAM) role authentication to access your Amazon S3 server, select this option. When you use this feature, leave the Access key ID and the Secret access key properties blank. The resource manager accesses the Amazon Simple Storage Service server by using temporary credentials that it gets from the Amazon Elastic Compute Cloud instance metadata.

   To use this feature:

   • The resource manager, the IBM Content Manager API, and the system administration client must be Version 8.7 or later.
   • The resource manager application must be deployed and run in an Amazon Elastic Compute Cloud (EC2) instance with IAM role attached. The IAM role must have permission to access your Amazon S3 buckets. For more information about how to configure this, see the Amazon documentation.

   Enable virtual hosted-style requests

   This feature is available in V8.7 fix pack 1 and later.

   To use a virtual hosted-style URL access type hostname, select this option, and enter a virtual hosted-style hostname in the Hostname field.

   For Microsoft Azure Block Blob Storage: Do not select this option, and make sure that you specify a path-style URL access type hostname in the Hostname field.

5. Click OK.

   IBM Content Manager creates the server definition.

6. Define the Device Manager: Right-click Device Managers, and then click New.

   The Device Manager Properties dialog box opens.

7. Specify the name of the device manager. This can be any name that you want, but it must be different from any other device managers that are defined on your system. For more information, see Creating a device manager.
8. In the Description field, do one of the following:
   • For Microsoft Azure Block Blob Storage, specify: Azure Block Blob Device Manager
   • For other cloud storage servers, specify a meaningful description, for example: IBM Cloud Object Storage Retention S3 Device Manager
9. In the Parameters field, type one of the following:
   • For IBM Cloud Object Storage with retention protection enabled:

     "mode=retention"

     For more information about IBM Cloud Object Storage with retention, see Support for IBM Cloud Object Storage retention.

   • For Amazon S3 with object lock, Dell EMC Elastic Cloud Storage with object lock, Hitachi Content Platform for Cloud Scale (HCP for cloud scale) with object lock, or IBM Cloud Object Storage with object lock:

     "mode=retention_s3"

     For more information, see Support for cloud object storage with object lock.

   • For other cloud storage servers: Leave the Parameters field blank.
10. In the Class field, type one of the following:
    • For Microsoft Azure Block Blob Storage: AzureBlockBlob
    • For other cloud storage servers: S3
11. Set Device Manager to Enable, and then click OK.
12. Define a storage class that uses the device manager that you have created. For more information, see Creating a storage class.
13. Define the Cloud Object Storage volume: Right-click Cloud Object Storage Volumes, and then click New.

    The New Cloud Object Storage dialog box opens.

    Note: The Export All to XML option is not available for cloud object storage volumes in V8.7 fix pack 1 and earlier.
14. In the Bucket field, type the name of the cloud storage bucket (or for Microsoft Azure Block Blob Storage, the storage container) that is to be used as the resource manager storage volume.
15. Assign the volume to the storage group that you want, and then click OK.

    For more information, see Creating a storage group.

16. Define a migration policy that uses the new storage class. For more information, see Creating a migration policy.

    If you are using IBM Cloud Object Storage with retention:

    • the storage class of the cloud object storage must be the only one or the last one in the migration sequence and
    • the retention period must be Forever.
17. Define a collection that uses the new migration policy. For more information, see Creating a collection.
18. Define an item type that uses the new collection. For more information, see Creating an item type.

    You can now use the item type to create, read, update, and delete objects in IBM Content Manager. Objects that use this item type are stored in the cloud storage bucket.