Converting IBM Storage Scale independent fileset to manual update mode

You can convert the existing IBM Storage Scale independent fileset to manual update (MU) mode.

Before you convert the IBM Storage Scale independent fileset to MU mode, ensure that the type of data such as file names, file sizes, and file types must match to the data syntax and semantics that are supported by cloud object storage (COS) provider.

To convert the IBM Storage Scale independent fileset to MU mode, the existing independent fileset with available data can be used. The mmafmcosconfig with --convert option is used to convert the fileset to MU mode.

After the fileset is converted to MU mode, all or specific data can be uploaded to the COS bucket.

Note:

With mmafmcosconfig --convert option, other options: --dir, --uid, --gid, and --perm are not supported.
While converting or creating MU mode fileset, mmafmcosconfig –cleanup options must not be used.

The following example contains the steps that are used to convert IBM Storage Scale independent fileset to MU mode:

Create IBM Storage Scale independent fileset and link that fileset.


Node1 03:10:30 1] mmcrfileset fs1 specfileset --inode-space=new
Fileset specfileset created with id 11 root inode 3145731.
Node1 03:10:40 1] 
Node1 03:10:47 1]  mmlinkfileset fs1 specfileset -J /gpfs/fs1/specfileset
Fileset specfileset linked at /gpfs/fs1/specfileset

Create few data in the fileset.


Node1 03:10:48 1] echo "file data" > /gpfs/fs1/specfileset/file1
Node1 03:10:55 1] echo "file data" > /gpfs/fs1/specfileset/file2
Node1 03:10:57 1] echo "file data" > /gpfs/fs1/specfileset/file3

Set up keys from Cloud object storage for a new bucket called “bucketoncos”.

Node1 03:11:00 1]  mmafmcoskeys bucketoncos:s3.us-east.cloud-object-storage.appdomain.cloud set 779c2178d53a4497de8d6cdc72 ef08ac5bcc1b8c35a293a7fe24c2effe54d2beba0

Create MU fileset by using independent fileset “specfileset”.

Node1 03:12:26 1]  mmafmcosconfig  fs1 specfileset --endpoint http://s3.us-east.cloud-object-storage.appdomain.cloud  --new-bucket bucketoncos  --object-fs --mode mu --xattr --convert
mmafmcosconfig: Fileset specfileset is not an AFM fileset.
Converting GPFS fileset to AFM manual-updates fileset...
Fileset specfileset changed.
Node1 03:12:38 1] 
Node1 03:12:39 1]

Verify that converted fileset uses mmlsfileset command.


Node1 03:12:39 1] mmlsfileset fs1 specfileset --afm -L
Filesets in file system 'fs1':

Attributes for fileset specfileset:
====================================
Status                                  Linked
Path                                    /gpfs/fs1/specfileset
Id                                      11
Root inode                              3145731
Parent Id                               0
Created                                 Tue Jan 11 03:10:36 2022
Comment                                 
Inode space                             6      
Maximum number of inodes                100352
Allocated inodes                        100352
Permission change flag                  chmodAndSetacl
afm-associated                          Yes
Permission inherit flag                 inheritAclOnly
Target                                  http://s3.us-east.cloud-object-storage.appdomain.cloud:80/bucketoncos
Mode                                    manual-updates
File Lookup Refresh Interval            120
File Open Refresh Interval              120
Dir Lookup Refresh Interval             120
Dir Open Refresh Interval               120
Async Delay                             disable
Last pSnapId                            0
Display Home Snapshots                  no
Parallel Read Chunk Size                0
Number of Gateway Flush Threads         8
Prefetch Threshold                      0 (default)
Eviction Enabled                        yes (default)
IO Flags                                0x10080000 (afmObjectXattr,afmMUPromoted)
IO Flags2                               0x0 (default)

Files available on the fileset.

Node1 03:12:58 1] ls -sh  /gpfs/fs1/specfileset
total 1.5K
512 file1  512 file2  512 file3
Node1 03:13:16 1]

Create an objectlist file and upload the objects.

Node1 03:48:31 1] cat  /root/objectlist
/gpfs/fs1/specfileset/file1
/gpfs/fs1/specfileset/file2

Node1 03:48:41 1] mmafmcosctl fs1 specfileset /gpfs/fs1/specfileset/ upload --object-list /root/objectlist
     Queued    (Total)      Failed                TotalData
                                              (approx in Bytes)
       2         (2)           0                       20 
Object Upload successfully queued at the gateway.
Node1 03:48:50 1]

Check the files stat by using –upload-stats.

Node1 03:49:12 1] mmafmcosctl fs1 specfileset --upload-stats
mmafmcosctl: Statistics of last or currently running upload are as follows:
Fileset Name    Pending         Failed          Total           Total data(Bytes)    Throughput(KB/s)
------------    ------------    ------------    ------------    -------------------- ---------------
specfileset     0               0               2               20                   0              
Node1 03:49:27 1]