Introduction to Cloud services
Transparent cloud tiering is a feature of IBM Spectrum Scale that provides a native cloud storage tier.
- Transparent cloud tiering
- Cloud data sharing
Transparent cloud tiering allows data center administrators to free up IBM Spectrum Scale storage capacity, by moving out cooler data to the cloud storage, reducing capital and operational expenditures. Tiering can also be used to archive an extra copy of your data by using pre-migration, a function that copies data rather than moving it. The Transparent cloud tiering feature leverages the existing ILM policy query language semantics available in IBM Spectrum Scale, and administrators can define policies to tier data to cloud storage. On an IBM Spectrum Scale cluster with multiple storage tiers configured, this external cloud storage can be used as the cooler storage tier to store infrequently accessed data from a cool storage pool. For performance reasons, it is recommended not to move any active or hot data to this external storage pool, as it drives excessive data traffic on the Transparent cloud tiering which in turn can cause delays, leading to problems like application timeouts. Copying hot data to the cloud by using the pre-migration function is acceptable when the file is not likely to be updated or deleted soon.
Cloud data sharing is an IBM Spectrum Scale Cloud services that allows a way to set up sharing between IBM Spectrum Scale and various types of object storage, including IBM Cloud® Object Storage. Furthermore, for export of data to object storage the service can be invoked from an ILM policy to allow for periodic sharing of data based on when the policy is run. For import, you can specify a list of files that you can use to move object storage (by using your IBM® Cloud data sharing service) into the IBM Spectrum Scale cluster.
This is useful when data needs to be distributed across multiple domains or when large amounts of data need to be transferred.
You can associate your Cloud services file systems and file sets with a Cloud services node group. Up to four node groups are supported in a cluster to allow for scaling (of the solution). For each file system or file set, you can associate up to two cloud storage tiers. For example, you could define an IBM Cloud Object Storage service locally to act as your cool tier. You could use IBM's Public Cloud Object Storage (on IBM Cloud) for your cold tier. The following diagram is an example of a Cloud services configuration that is on a cluster:
Figure 1 illustrates these features.
- For performance reasons, the median file size to be migrated to the cloud tier must be greater than 1 MB. Migration is supported for file size less than one MB, but performance is slower because of the overhead that is associated with small files.
- For Transparent cloud tiering, data on the Cloud Object Storage is opaque and cannot be accessed directly by applications. All I/O operations must happen through your IBM Spectrum Scale system.
- Transparent cloud tiering works with IBM Spectrum Scale on multi-site stretch clusters to allow continued access to cloud storage data even if failure of an entire site.
- For applications that require tiering of data that scales beyond 100 million files per file
system, you can create extra containers on cloud storage to hold your files. For applications that
require more I/O bandwidth than a single maximum node group (four nodes) is capable of, you can
create extra node groups. Note: You must create a new container when the existing container has approximately 100,000,000 files.
- Unsupported use cases
- Transparent cloud
tiering does not support the following use cases:
- Using Transparent cloud tiering to migrate or recall hot (active) data.
- Using Transparent cloud tiering as a backup mechanism.
- Using Transparent cloud tiering to restore data in disaster recovery scenarios.
Note: IBM Spectrum Protect cloud-container storage pools can be used to back up data to cloud storage providers.