Connectivity to cloud object storage

Each AFM to cloud object storage fileset in a cluster is served by one of the nodes that is designated as a gateway in the cluster.

The gateway node that is mapped to a fileset is called the primary gateway of the fileset. The primary gateway acts as the owner of the AFM fileset and communicates with the cloud object storage by using the mmafmtransfer command. An IBM Storage Scale cluster that has many AFM to cloud object storage filesets can have multiple gateway nodes. These gateway nodes connect to the cloud object storage by using HTTP or HTTPS protocols.

Target backend

AFM to cloud object storage can communicate with different cloud object storage backend. Some cloud object storage uses different API other than S3 standard API for the following reasons:

To enable an AFM fileset to synchronize objects with different cloud to object storage backend such as Microsoft Azure Blob, Google GCP or AWS VHB style.
To enable AFM to synchronize with specific cloud object storage backend, AFM to cloud object fileset must be created with a specified parameter.

The options are as follows:

--azure: Providing this option during the creation of an AFM to cloud object storage fileset by using the mmafmcosconfig command enables AFM to use Microsoft Azure Blob storage specific API.
--gcs: Providing this option during the creation of an AFM to cloud object storage fileset by using the mmafmcosconfig command enables AFM to use Google cloud object storage specific API.
--vhb: Provides support of S3 style virtual hosting of bucket such as URL to be used during the creation of an AFM to cloud object storage fileset. For VHB, the format of endpoint is such as https://bucket-name.s3.region-code.amazonaws.com. AFM requires a region to be added to the URL and separated it by using ‘@’ symbol.

An example is as follows:

Add the key by issuing the following command:

 mmafmcoskeys regbkt1**:**ap-south-1@regbkt1.s3.amazonaws.comset akey1 skey1

Verify that the key was added successfully by issuing the following command:

 mmafmcoskeys regbkt1:ap-south-1@regbkt1.s3.amazonaws.com get

A sample output is as follows:

regbkt1:ap-south-1@regbkt1.s3.amazonaws.com=COS:akey1:skey1

Create an AFM to cloud object storage fileset by issuing the following command:
```
**#**mmafmcosconfig fs1 iw1 --endpoint https://ap-south-1@regbkt1.s3.amazonaws.com --object-fs --xattr --bucket regbkt1 --mode iw --acls --vhb --directory-object
```
Note: To enable AFM to cloud object storage for standard other S3 cloud object storage backends, above options are not needed. Default option implicitly enables AFM to use Standard S3 APIs.

Underlying protocol

Communication between IBM Storage Scale and a cloud object storage can be enabled by using HTTP or HTTPS.

The AFM to cloud object storage also enables sharing of data across AFM to cloud object storage clusters and a cloud object storage, even if the networks are unreliable or have high latency by using the built-in queuing optimization techniques of AFM. For more information, see Active File Management.

Note:

Within a single IBM Storage Scale cluster where AFM to cloud object storage is enabled, application nodes comply with POSIX semantics. The file or object locking across cache and a cloud object storage is not supported.

When you perform operations on AFM to cloud object storage filesets, ensure that the operations are supported on a cloud object storage over the chosen protocol. Because the operations that are performed from an AFM to cloud object storage fileset are replayed on the cloud object storage as normal object operations. Ensure that the bucket restrictions and limitations from a cloud object storage provider are followed.

The cache state of an AFM to cloud object storage is similar to an AFM or AFM-DR fileset. You can check the state by using the mmafmctl getstate command. This command is used to check the status of synchronization of the specified fileset. For example, if a bucket is not accessible on the cloud object storage server, the fileset is moved to the Unmounted state and the following log error is displayed:

2020-08-27_10:32:01.571-0400: [E] AFM: COS access of 192.168.1.1://bucket1 failed with error 13 for file system fs1 fileset ID 9. Caching will be disabled and the access will be tried again after 300 seconds, on the next request to the gateway.

When the bucket is accessible, AFM to cloud object storage tries the synchronization again. Similarly, other states become valid with the AFM to cloud object storage fileset.

For more information about AFM fileset states, see Monitoring AFM and AFM DR.