Fileset operation modes

This mode allows multiple AFM to cloud object storage filesets to point to the same cloud object storage bucket. Multiple AFM to cloud object storage filesets can be on the same IBM Storage Scale cluster or on a different cluster and points to a cloud object storage. There is no synchronous locking between clusters while updating objects on a cloud object storage. Each AFM to cloud object storage fileset reads from a cloud object storage and makes updates to the cloud object storage independently based on the revalidation intervals and asynchronous delay.

This mode is used to access different objects from each IW AFM to cloud object storage site, for example, unique users at each site that update objects in their cloud object storage bucket. This mode allows multiple AFM to cloud object storage clusters to modify the same set of objects. Only advanced users must do this modification because there is no locking or ordering between updates. Updates are propagated to a cloud object storage in an asynchronous manner and can be delayed because of network disconnections. Therefore, conflicting updates from multiple AFM to cloud object storage sites can cause the data on the cloud object storage site to be undetermined.

This mode is similar to the RO mode, although you can create and modify objects in an AFM to cloud object storage fileset. Updates in the AFM to cloud object storage fileset are considered local to the AFM to cloud object storage fileset and get decoupled from the corresponding object on a cloud object storage. Local updates are never pushed back to a cloud object storage.

When an object is downloaded and modified in an AFM to cloud object storage LU-mode fileset, the modified data does not synchronize back with the bucket on the cloud object storage server even if the object on the bucket is modified. This behavior is similar to the AFM LU-mode fileset. For more information about AFM modes, see Operations with AFM modes.

The manual update (MU) mode supports manual replication of the files or objects by using ILM policies or user-provided object list. MU mode is supported on AFM to cloud object storage backend. The IBM Storage Scale independent fileset can be converted to MU mode fileset or MU mode fileset can be created as a new relationship to cloud object storage.

MU mode fileset provides the flexibility to upload and download files or objects to and from cloud object storage after you finalize the set of objects to upload or download. Unlike other AFM to cloud object storage objectfs fileset modes, MU mode depends on manual intervention from administrators to upload and download the data to be in sync. As administrators, you can also automate upload and download by using ILM policies to search specific files or objects to upload or download.

MU mode semantics:

• Applications or users create data in the MU mode fileset. The data is not transferred to cloud object storage backend automatically. The data can be dirty or hot data.
• Either all the files or specific files can be determined and uploaded by using the mmafmcosctl upload command. Or, policies can help determine these specific files.
• When the files are added to cloud object storage or the data is changed on cloud objects storage, these files can be downloaded by using the mmafmcosctl download command.
  Important: MU mode does not recognize the files added to cloud object storage or modified data automatically, to download these files or data administrators can create object list that contains the names of the files from cloud object storage.
• Metadata is refreshed only once. This is applicable only if the MU fileset is created pointing to non-empty bucket. Each file is refreshed before allowing the update if readdir was not performed.
• All the changes to files are local to the fileset except remove operations, which can be made automatically replicated by using the config option.

Deleting files from MU mode fileset

Files on MU mode fileset take up inode and space from fileset. Files can be deleted from MU mode fileset but the file inodes are reclaimed from the fileset inode space only when mmafmcosctl delete commands are issued. MU mode has two delete options backed by mmafmcosctl delete command. The files can be deleted locally by using posix commands. For example, rm. MU mode does not reclaim inodes immediately after the deletion until the delete operation is executed to the COS target.

The mmafmcosctl delete commands – from cache option looks for deleted files in the MU fileset and reclaim the file inodes, whereas –from-target option looks for the deleted files in MU fileset, reclaims the inodes and queue the delete operation to COS so that files deleted from MU fileset gets deleted on COS as well.

Note: After the delete --from-cache is performed, the inodes are reclaimed from MU mode fileset. After execution of this command if the delete --from-target is executed, the deletes are not queued on the cloud object storage.

Example : Deleting and reclaiming inodes from MU mode fileset

1. Create MU mode fileset.

   node1 1] mmafmcosconfig fs1 mufilesetdemo  --endpoint http://s3.us-east.cloud-object-storage.appdomain.cloud --bucket  demobucketx  --object-fs --mode mu  
   node1 1] 
   node1 1] 
   node1 1] mmafmctl fs1 getstate
   Fileset Name    Fileset Target                                Cache State          Gateway Node    Queue Length   Queue numExec 
   ------------    --------------                                -------------        ------------    ------------   ------------- 
   mufilesetdemo   http://s3.us-east.cloud-object-storage.appdomain.cloud:80/demobucketx Inactive
   node1 1]
   

2. Check the used inodes.

   node1 24:16 1] mmlsfileset fs1 mufilesetdemo -i
   Collecting fileset usage information ...
   Filesets in file system 'fs1':
   Name                     Status    Path                                         InodeSpace      MaxInodes    AllocInodes     UsedInodes
   mufilesetdemo            Linked    /gpfs/fs1/mufilesetdemo                         3               100352         100352              3
   node1 1]
   

3. Create five files.

   node1 1] echo "filedata"  > /gpfs/fs1/mufilesetdemo/file1
   node1 1] echo "filedata"  > /gpfs/fs1/mufilesetdemo/file2
   node1 1] echo "filedata"  > /gpfs/fs1/mufilesetdemo/file3
   node1 1] echo "filedata"  > /gpfs/fs1/mufilesetdemo/file4
   node1 1] echo "filedata"  > /gpfs/fs1/mufilesetdemo/file5
   

4. Check inode stats : Here inode numbers are increased by five.

   node1 1] mmlsfileset fs1 mufilesetdemo -i
   Collecting fileset usage information ...
   Filesets in file system 'fs1':
   Name             Status    Path                     InodeSpace    MaxInodes  AllocInodes     UsedInodes
   mufilesetdemo    Linked    /gpfs/fs1/mufilesetdemo   3            100352      100352              8
   node1  1]
   

5. Create an object list with all files and upload.

   node1  1] cat /root/objectlist
   /gpfs/fs1/mufilesetdemo/file1
   /gpfs/fs1/mufilesetdemo/file2
   /gpfs/fs1/mufilesetdemo/file3
   /gpfs/fs1/mufilesetdemo/file4
   /gpfs/fs1/mufilesetdemo/file5
   node1  1] 
   node1  1] 
   node1  1] mmafmcosctl fs1 mufilesetdemo /gpfs/fs1/mufilesetdemo upload --object-list /root/objectlist
        Queued    (Total)      Failed                TotalData
                                                 (approx in Bytes)
            5       (5)           0                       45 
   Object Upload successfully queued at the gateway.
   node1  1] 
   

6. Remove all the files from the MU fileset.

   node1  1] rm -rf /gpfs/fs1/mufilesetdemo/file*
   node1  1]
   

7. Check the inodes stats : inodes are not yet reclaimed from MU fileset.

   node1  1]  mmlsfileset fs1 mufilesetdemo -i
   Collecting fileset usage information ...
   Filesets in file system 'fs1':
   Name                     Status    Path                      InodeSpace      MaxInodes    AllocInodes     UsedInodes
   mufilesetdemo            Linked    /gpfs/fs1/mufilesetdemo    3               100352         100352             8
   node1 1]
   

8. Issue the following command to reclaim the inodes from MU mode fileset:

   node1 1] mmafmcosctl fs1 mufilesetdemo /gpfs/fs1/mufilesetdemo/  delete --from-cache

9. Check the inode stats. Here the used inodes are reclaimed.

   node1 1] mmlsfileset fs1 mufilesetdemo -i
   Collecting fileset usage information ...
   Filesets in file system 'fs1':
   Name                     Status    Path                       InodeSpace      MaxInodes    AllocInodes     UsedInodes
   mufilesetdemo            Linked    /gpfs/fs1/mufilesetdemo     3               100352         100352              3
   node1 1]
   

Note: The mmafmcosctl command with option delete --from-cache, does not queue deletes to cloud object storage.

Example: Deleting files, reclaiming inode, and queueing deletes to COS

1. Check MU mode fileset inode stats.

   Node1 1] mmlsfileset fs1 mufilesetdemo -i
   Collecting fileset usage information ...
   Filesets in file system 'fs1':
   Name                     Status    Path                      InodeSpace    MaxInodes    AllocInodes     UsedInodes
   mufilesetdemo            Linked    /gpfs/fs1/mufilesetdemo    3            100352         100352              3
   Node1 1]
   

2. Create five files in fileset1.

   Node1 1]  echo "filedata"  > /gpfs/fs1/mufilesetdemo/file1
   Node1 1]  echo "filedata"  > /gpfs/fs1/mufilesetdemo/file2
   Node1 1]  echo "filedata"  > /gpfs/fs1/mufilesetdemo/file3
   Node1 1]  echo "filedata"  > /gpfs/fs1/mufilesetdemo/file4
   Node1 1]  echo "filedata"  > /gpfs/fs1/mufilesetdemo/file5
   Node1 1]
   

3. Check the inodes stats:Used inodes increased by five.

   Node1 1] mmlsfileset fs1 mufilesetdemo -i
   Collecting fileset usage information ...
   Filesets in file system 'fs1':
   Name                     Status    Path                       InodeSpace      MaxInodes    AllocInodes     UsedInodes
   mufilesetdemo            Linked    /gpfs/fs1/mufilesetdemo     3               100352         100352              8
   Node1  1]
   

4. Create an object list and upload all the files to COS.

   Node1  1] 
   Node1  1]  mmafmcosctl fs1 mufilesetdemo /gpfs/fs1/mufilesetdemo upload --object-list /root/objectlist
        Queued    (Total)      Failed                TotalData
                                                  (approx in Bytes)
            5       (5)         0                      45 
   Object Upload successfully queued at the gateway.
   Node1 1]
   

5. Remove the files from MU mode fileset.

   Node1 1] rm -rf /gpfs/fs1/mufilesetdemo/file*
   Node1 1]
   

6. Check the inode usage : Inodes are not reclaimed after delete.

   Node1 1] mmlsfileset fs1 mufilesetdemo -i
   Collecting fileset usage information ...
   Filesets in file system 'fs1':
   Name                     Status    Path                        InodeSpace      MaxInodes    AllocInodes     UsedInodes
   mufilesetdemo            Linked    /gpfs/fs1/mufilesetdemo      3               100352         100352              8
   Node1 1]
   

7. Issue mmafmcosctl delete command with –from-target option.

   Node1 1] mmafmcosctl fs1 mufilesetdemo /gpfs/fs1/mufilesetdemo/  delete --from-target

8. Check that the remove operations are queued and executed to cos.

   Node1 1] mmafmctl fs1 getstate
   Fileset Name    Fileset Target                                Cache State          Gateway Node    Queue Length   Queue numExec 
   ------------    --------------                                -------------        ------------    ------------   ------------- 
   mufilesetdemo   http://s3.us-east.cloud-object-storage.appdomain.cloud:80/demobucketx Dirty c7f2n04 5             22           
   Node1 1] mmafmctl fs1 getstate
   Fileset Name    Fileset Target                                Cache State          Gateway Node    Queue Length   Queue numExec 
   ------------    --------------                                -------------        ------------    ------------   ------------- 
   mufilesetdemo   http://s3.us-east.cloud-object-storage.appdomain.cloud:80/demobucketx Active c7f2n04 0            27           
   Node1 1]
   

9. Check that the inodes in MU fileset are reclaimed.

   Node1 1] mmlsfileset fs1 mufilesetdemo -i
   Collecting fileset usage information ...
   Filesets in file system 'fs1':
   Name                     Status    Path                       InodeSpace      MaxInodes    AllocInodes     UsedInodes
   mufilesetdemo            Linked    /gpfs/fs1/mufilesetdemo     3               100352         100352              3
   Node1 1]
   

MU mode autoremove

To keep the deletes in sync with COS, the autoremove feature can be configured on the MU fileset by using mmchfileset command. When the parameter afmMUAutoRemove is set to yes then removes are automatically queued to the cloud object storage.

For setting this afmMUAutoRemove parameter option first, do the following:

1. Stop the fileset by using mmafmctl stop option.
2. Set the parameter on the fileset by using mmchfileset <fs> <fileset> -p afmMuAutoRemove=yes.
3. Again start the fileset by using mmafmctl start command.

Note: When the auto remove option is configured the inodes are reclaimed from MU mode fileset automatically.

Use case scenarios for MU mode:

1. The MU mode supports scenarios where the data changes at AFM to cloud object storage fileset is not needed to be in sync with the target cloud object storage. Applications can perform data modification at fileset level and whenever it is necessary to sync the data to the cloud object storage that can be synced.

2. Similarly, application running on cloud can modify the data on cloud and when it is required to be in sync with AFM to cloud object storage MU mode fileset, the data can be downloaded. This helps in bandwidth use as data can be uploaded or downloaded at specific times. For example, application works on files in the MU fileset and uploaded to COS at later time in the day when the load on the network is less.