Migration of a transparent cloud tiering-enabled IBM Storage Scale fileset or file system to an AFM to cloud object storage fileset in the manual update mode

AFM can perform inline migration of a transparent cloud tiering (TCT) enabled IBM Storage Scale-independent fileset or file system by promoting it to an AFM to cloud object storage fileset in the manual update (MU) mode. During the migration, file data is not recalled from a cloud tier to a cache fileset. AFM replaces TCT attributes from all files to AFM-specific attributes after the conversion.

When TCT is migrating data and metadata of a file to a cloud tier, local data blocks of the file are evicted from the IBM Storage Scale file system. TCT can recall this file from the cloud so that the data blocks are available on both sides.

States of files in a TCT-enabled fileset

In a TCT-enabled fileset, files can have the following states:
Resident
File data is local to a file system and is not migrated to the cloud.
Non-Resident
File data is not local to a file system (evicted) and is migrated to the cloud.
Co-Resident
File data is recalled from the cloud, and file data is local to a file system and both side of the cloud.

AFM processes tiered files in the Non-Resident or Co-Resident state to the cloud object storage. Also, it converts them to an AFM migrated file by removing TCT attributes and replaces TCT attributes with AFM attributes. An existing TCT-enabled fileset or a file system can be promoted to an MU-mode fileset by issuing the mmafmcosconfig --convert command.

During the promotion, AFM queries all the files in the Co-Resident or Non-Resident state in a TCT-enabled fileset by running a policy. It lists the list-file and processes it.

  • AFM removes TCT information on the cloud location, and replaces it with AFM style object information (actual file path).
  • AFM removes TCT attributes from the file on the cache side and sets AFM attributes.
  • Synchronizes both cache and cloud-side files data and metadata.
  • If the file is in the Non-resident or Co-resident state, AFM keeps data without changing it. For a Co-resident state file, you need not to recall file data from the cloud after the promotion. After the promotion is completed, you can perform all operations seamlessly on the AFM to cloud object storage fileset in the MU mode such as upload, download, or evict.
Before the promotion of a TCT fileset, mmlsattr -X -n < AttributeName> <file> shows the following TCT attribute. This attribute is removed after the conversion. 
# mmlsattr -X  -n dmapi.MCEA /gpfs/fs1/tct3/fl1.txt
A sample output is as follows:
file name:      /gpfs/fs1/tct3/fl1.txt
Files in the Non-resident or Co-resident state in a TCT fileset show the following information on the cloud object storage:
002FC5684DD239F6.41DBE301648AD5F2.66DD8A640A006468.0000000000000000.3273D35B.0000000000140134/0000000000000001.DATA
 
002FC5684DD239F6.41DBE301648AD5F2.66DD8A640A006468.0000000000000000.3273D35B.0000000000140134/0000000000000001.META

After the promotion of a TCT-enabled fileset to an AFM MU fileset, AFM removes TCT style format and replaces it with an AFM supported bucket or object style format.

Migration considerations

  • Migration of a TCT-enabled fileset or file system is only for the tiering cloud-service-type. It is not supported for sharing Cloud-Service-Type.
  • Before conversion, ensure that a TCT fileset does not have any file or data structure that an AFM to cloud object storage fileset in the MU mode does not support. AFM to cloud object storage limitations such as dependent fileset, file clone not supported. For more information about the AFM limitations, see AFM to cloud object storage limitations.
  • AFM converts metadata of only TCT-enabled files to AFM supported file metadata because TCT supports tiering of only files and not directories. AFM does not synchronize the directory metadata.
  • Before promotion, directories with metadata set are available only at the fileset site. If you want to upload directory metadata to the cloud tier, promote it by using the --directory-object option and upload it by issuing the mmafmcosctl upload command after the promotion.
  • Symlinks, which were migrated to the cloud tier, are not supported during the migration. Upload symlinks manually after the promotion by issuing the mmafmcosctl upload command.
  • All resident files, such as files that TCT did not tier to a cloud, remain only at an IBM Storage Scale location. After the promotion, you can upload these files by using the mmafmcosctl upload command.