Enabling AFM caching

IBM Storage Scale active file management (AFM) to cloud storage enables automatic tiering of application data to Amazon Simple Storage Service (Amazon S3) or Google Cloud Storage (GCS).

For a detailed explanation of the AFM feature, see Introduction to AFM to cloud object storage.

If the cloud object storage target is AWS S3, the cloudkit simplifies the authentication setup that is needed by using the AWS instance IAM profile. Whereas if the target is GCS, the service account that is chosen during the configuration is used to generate the HMAC credentials, which are further used to setup authentication to the GCS bucket from gateway nodes.

To enable AFM to cloud object storage, use the ./cloudkit caching setup command. The following steps illustrate a simplified setup of AFM caching.

  1. Deploy the AFM gateway nodes.

    Use the ./cloudkit edit cluster command to provision or add AFM gateway nodes to an existing storage cluster or combined cluster.

    Example:
    # ./cloudkit edit cluster
I: Logging at /root/scale-cloudkit/logs/cloudkit-23-7-2024_6-28-28.log
? Are you operating cloudkit in air-gapped or no-internet environment:  No
? Select a cluster:  scale-demo-cls | None | us-central1 | GCP
? Select the role: Gateway-only
I: Opted cluster 'scale-demo-cls' currently has '0' gateway nodes.
? IBM Storage Scale Active File Management (AFM) Gateway instance type:  n1-standard-2  | vCPU(2)  | RAM (7.50 GB) | Egress Network Bandwidth (Up to 10 Gbps)
? IBM Storage Scale Active File Management (AFM) Gateway node count:  1
I: Requested to expand the gateway nodes by '1' node(s).
? Do you wish to view cost before provisioning the resources (this requires infracost api_key):  (y/N)
  2. Configure AFM to cloud object storage.

    To enable an AFM relationship, issue the ./cloudkit caching setup command.

    Example:
    ]# ./cloudkit setup caching
I: Logging at /root/scale-cloudkit/logs/cloudkit-23-7-2024_6-50-1.log
? Are you operating cloudkit in air-gapped or no-internet environment:  No
? Select a storage cluster:  scale-demo-cls | None | us-central1 | GCP
? Storage cluster management GUI username:  administrator
? Storage cluster management GUI password:  [? for help] ***********
? IBM Storage Scale Active File Management caching target type:  Cloud-Object-Storage
? Fileset name:  afmcos_fset
? IBM Storage Scale Active File Management caching mode:  Single-writer | sw | One cache fileset does all the writing
? AFM caching target name:  Select this option for custom input.
? AFM Cloud Object Storage (bucket) name:  scale-objectstore
I: Bucket 'scale-objectstore' has been created.
? Select filesystem mount point:  /gpfs/fs1