Setting up a cloud services cluster

This topic provides step-by-step instructions for quickly setting up a cloud services node class that can support transparent cloud tiering or cloud data sharing on the IBM Storage Scale cluster.

In the multi-node setup, a maximum of 4 of any combination of CES or NSD nodes are supported as a cloud services node group -- a minimum of two are recommended for high availability. Data migration or recall workload is shared across all the nodes. During a transparent recall or migration, the ILM policy gathers a list of files to be migrated, and the list is divided equally among all the nodes.

Note: Once a multi-node setup is up and running, the transparent cloud tiering GUI components are available on the IBM Storage Scale GUI.

The mtime attribute on the IBM Storage Scale file system indicates when a file is updated. For a cloud services managed file system, ensure that the mtime parameter is set to yes, so that when cloud services internally invokes stat() or fstat() calls during migration, file system reports exact mtime value for a file. If mtime parameter is set to "no", there is a potential data loss situation, when a modified / updated co-resident file does not get migrated as a new version to the cloud. You can check the mtime status by using the mmlsfs <file system> -E command.

Perform the following steps for setting up a cloud services cluster:

  1. Install the cloud services RPMs on the required IBM Storage Scale nodes. For more information, see Installing transparent cloud tiering (discontinued).
  2. Designate the nodes where you installed the RPMs as cloud services nodes. For more information, see Designating the cloud services nodes.
  3. Start cloud services on all nodes. For more information, see Starting up the cloud services software.
  4. Create a cloud account. For simpler configurations, use the built-in key manager that comes with cloud services. Use KLM when KLM is already running on IBM Storage Scale (for other reasons) or if specific security requirements that drive its usage. For more information, see Managing a cloud storage account.
  5. Define cloud storage access points (CSAP). For more information, see Defining cloud storage access points (CSAP).
  6. Create a cloud services. For more information, see Creating cloud services.
  7. Configure a key manager. For more information, see Configuring cloud services with SKLM (optional).
  8. Create a container pair set and test your configuration. For more information, see Binding your file system or fileset to the Cloud service by creating a container pair set.
  9. Make sure you back up your configuration and your key manager. For more information, see Backing up the cloud services configuration.
.

If you are doing transparent cloud tiering, do the following steps to verify that the cluster is ready for transparent recall/migration based on an ILM policy:

  • Create a sample policy, migrate.policy, and apply this policy by using the cloud services nodes by with the command: mmapplypolicy gpfs -P migrate.policy -N FirstCloud
    Note: If you administer the defaultHelperNodes settings, you do not need to specify the -N parameter to distribute workload here.
You can confirm that data migration is happening across multiple nodes by looking at the following events:
  • Each transparent cloud tiering node has policy files under /tmp.
  • Each transparent cloud tiering node starts reflecting the migration metrics for the amount of data that is passed through it.

cloud services run maintenance automatically in maintenance windows in the background. Do not forget to review the maintenance windows and adjust them appropriately. For more information, see Configuring the maintenance windows.

Note: If you change the IP address or network settings of any of the cloud services nodes once it is operational, do the following steps:
  1. Stop the cloud services (by using the mmcloudgateway service stop command), the way it is mandated to stop IBM Storage Scale services while you change the node IP address.
  2. To change the IP address, it is recommended to follow the steps that are provided in Changing IP addresses or host names of cluster nodes.

For more information, see Configuring and tuning your system for cloud services.