Configuring a new file system for the replication by using the manual updates mode of the AFM to cloud object storage

A new file system can be created and configured for replicating data by using the manual update mode of the AFM to cloud object storage.

Plan and provision disks to create a new file system.
Create a stanza to create an NSD.
Create NSDs by using these disks.
Plan a storage and configure a cloud account for creating or accessing a cloud bucket, determine the secret and access keys, and create a bucket on cloud.
Use the mmafmcoskeys to add the keys for a bucket in an IBM Storage Scale cluster.
Create a file system by using the mmcrfs command with the afmTarget and afmMode parameters.
For more information about the parameters, see the Parameters for a new file system section.
Use the mmchfileset command to configure different options mentioned in the following example.

Note: If you want to set a UID and a GID on a file system, use the mmafmlocal chown $uid:$gid filesystempath command.

An example of configuring a new file system for the replication

Create a stanza.

# cat nsd
%nsd:
device=/dev/sdi
nsd=nsd1
servers=Node1,Node2
usage=dataAndMetadata
failureGroup=-1
pool=system
thinDiskType=auto
%nsd:
device=/dev/sdk
nsd=nsd2
servers=Node1,Node2
usage=dataAndMetadata
failureGroup=-1
pool=system
thinDiskType=auto
%nsd:
device=/dev/sdn
nsd=nsd3
servers=Node1,Node2
usage=dataAndMetadata
failureGroup=-1
pool=system
thinDiskType=auto
%nsd:
device=/dev/sdp
nsd=nsd4
servers=Node1,Node2
usage=dataAndMetadata
failureGroup=-1
pool=system
thinDiskType=auto

Create NSDs by using the created stanza.

 mmcrnsd -F nsd -v no
mmcrnsd: Processing disk sdi
mmcrnsd: Processing disk sdk
mmcrnsd: Processing disk sdn
mmcrnsd: Processing disk sdp
mmcrnsd: Propagating the cluster configuration data to all
affected nodes. This is an asynchronous process.

# mmlsnsd
File system   Disk name       NSD servers                                    
--------------------------------------------------
 (free disk)   nsd1            Node1,Node2          
 (free disk)   nsd2            Node1,Node2          
 (free disk)   nsd3            Node1,Node2          
 (free disk)   nsd4            Node1,Node2

Create a bucket and provision for space on cloud object storage providers by using consoles, and determine access and secret keys.
A bucket, city1, is created on IBM Cloud Object Storage.

Add the keys by issuing the mmafmcoskeys command.

 mmafmcoskeys city1:s3.us-east.cloud-object-storage.appdomain.cloud set 75fd1fa5c9e24aaea5093d198c0fa 61967eb3dfa2c66963bbc7d4e4725b14aa2aa10c4aff8

Create a file system, fs1, for the replication by using the city1 bucket and the MU mode.

 mmcrfs fs1 "nsd1;nsd2" -T /gpfs/fs1/ -p afmtarget=https://s3.us-east.cloud-object-storage.appdomain.cloud:443/city1 -p afmMode=manual-updates

The following disks of the fs1 file system are formatted on the c7f2n03 node:

nsd1: size 1142359 MB
nsd2: size 1142359 MB
Formatting file system ...
Disks up to size 8.82 TB can be added to storage pool system.
Creating Inode File
71 % complete on Thu May 18 07:48:29 2023
100 % complete on Thu May 18 07:48:31 2023
Creating Allocation Maps
Creating Log Files
Clearing Inode Allocation Map
Clearing Block Allocation Map
Formatting Allocation Map for storage pool system
Completed creation of file system /dev/fs1.
mmcrfs: Propagating the cluster configuration data to all
affected nodes. This is an asynchronous process.

Set required parameters. For more information, see the Parameters for a new file system.

 mmchfileset fs1 root -p afmObjectXattr=yes
Fileset root changed.
c7f2n02 18May07:50:16 0] mmchfileset fs1 root -p afmObjectDirectoryObj=yes
Fileset root changed.
c7f2n02 18May07:50:25 0] mmchfileset fs1 root -p afmObjectACL=yes
Fileset root changed.

Mount the file system on all nodes.

 mmmount fs1 -a
Thu May 18 07:51:27 EDT 2023: mmmount: Mounting file systems ...

List the file system attributes.

 mmlsfileset fs1 root --afm -L
Filesets in file system 'fs1':

Attributes for fileset root:
==================================
Status                                  Linked
Path                                    /gpfs/fs1
Id                                      0
Root inode                              3
Parent Id                               --
Created                                 Thu May 18 07:48:34 2023
Comment                                 root fileset
Inode space                             0      
Maximum number of inodes                2285568
Allocated inodes                        501760
Permission change flag                  chmodAndSetacl
afm-associated                          Yes
Permission inherit flag                 inheritAclOnly
Target                                  https://s3.us-east.cloud-object-storage.appdomain.cloud:443/city1
Mode                                    manual-updates
File Lookup Refresh Interval            30 (default)
File Open Refresh Interval              30 (default)
Dir Lookup Refresh Interval             60 (default)
Dir Open Refresh Interval               60 (default)
Async Delay                             disable
Last pSnapId                            0
Display Home Snapshots                  no
Number of Gateway Flush Threads         4
Prefetch Threshold                      0 (default)
Eviction Enabled                        yes (default)
IO Flags                                0x8280000 (afmObjectXattr,afmObjectDirectoryObj,afmObjectACL)
IO Flags2                               0x0 (default)

Create example data in the file system.

# mkdir /gpfs/fs1/dir1
c7f2n02 18May07:52:15 0] mkdir /gpfs/fs1/dir2
c7f2n02 18May07:52:22 0] echo 12345 > /gpfs/fs1/file1
c7f2n02 18May07:52:32 0] echo 12345 > /gpfs/fs1/file2
c7f2n02 18May07:52:34 0] echo 12345 > /gpfs/fs1/dir1/dfile1
c7f2n02 18May07:53:03 0] echo 12345 > /gpfs/fs1/dir2/dfile2

List the data.

ls -R /gpfs/fs1
/gpfs/fs1:
dir1 dir2 file1 file2
/gpfs/fs1/dir1:
dfile1
/gpfs/fs1/dir2:
dfile2
c7f2n02 18May07:53:30 0]

Upload or replicate the data to the city1 bucket.

 mmafmcosctl fs1 root /gpfs/fs1/ upload --all
 Queued         Failed                TotalData
                                  (approx in Bytes)
 6               0                       24 

Object Upload successfully queued at the gateway.

Check the status of upload.

# mmafmctl fs1 getstate 
Fileset Name    Fileset Target                                                     Cache State          Gateway Node    Queue Length   Queue numExec 
------------    ----------------------------------------------------------------   -------------        ------------    ------------   ------------- 
root            https://s3.us-east.cloud-object-storage.appdomain.cloud:443/city1  Active                c7f2n03             0              8

Check whether the data is replicated on the city1 bucket.

The following output is from the console of the cloud object storage:

Name      : dir1/
Date      : 0000-12-31 19:00:00 EST 
Size      : 0 B    
Type      : folder 

Name      : dir1/dfile1
Date      : 2023-05-18 07:55:25 EDT 
Size      : 6 B    
ETag      : d577273ff885c3f84dadb8578bb41399 
Type      : file 
Metadata  :
  X-Amz-Meta-Afm-Mode : 420 
  X-Amz-Meta-Afm-Uid  : 0 
  Content-Type        : binary/octet-stream 
  X-Amz-Meta-Afm-Gid  : 0 
  X-Amz-Meta-Afm-Atime: 2023-05-18T11:53:03Z 
  X-Amz-Meta-Afm-Mtime: 2023-05-18T11:53:03Z 

Name      : dir2/
Date      : 0000-12-31 19:00:00 EST 
Size      : 0 B    
Type      : folder 

Name      : dir2/dfile2
Date      : 2023-05-18 07:55:25 EDT 
Size      : 6 B    
ETag      : d577273ff885c3f84dadb8578bb41399 
Type      : file 
Metadata  :
  X-Amz-Meta-Afm-Atime: 2023-05-18T11:53:11Z 
  X-Amz-Meta-Afm-Mode : 420 
  X-Amz-Meta-Afm-Gid  : 0 
  X-Amz-Meta-Afm-Mtime: 2023-05-18T11:53:11Z 
  X-Amz-Meta-Afm-Uid  : 0 
  Content-Type        : binary/octet-stream 

Name      : file1
Date      : 2023-05-18 07:55:25 EDT 
Size      : 6 B    
ETag      : d577273ff885c3f84dadb8578bb41399 
Type      : file 
Metadata  :
  X-Amz-Meta-Afm-Gid  : 0 
  X-Amz-Meta-Afm-Atime: 2023-05-18T11:52:32Z 
  X-Amz-Meta-Afm-Mode : 420 
  X-Amz-Meta-Afm-Mtime: 2023-05-18T11:52:32Z 
  X-Amz-Meta-Afm-Uid  : 0 
  Content-Type        : binary/octet-stream 

Name      : file2
Date      : 2023-05-18 07:55:25 EDT 
Size      : 6 B    
ETag      : d577273ff885c3f84dadb8578bb41399 
Type      : file 
Metadata  :
  X-Amz-Meta-Afm-Atime: 2023-05-18T11:52:34Z 
  X-Amz-Meta-Afm-Gid  : 0 
  Content-Type        : binary/octet-stream 
  X-Amz-Meta-Afm-Mtime: 2023-05-18T11:52:34Z 
  X-Amz-Meta-Afm-Mode : 420 
  X-Amz-Meta-Afm-Uid  : 0

After the file system is created and configured, applications can create data in the new file system, and the data can be uploaded or downloaded to a cloud object storage manually or by using a policy with the mmafmcosctl reconcile command. For more information, see the AFM to cloud object storage policy based upload for manual updates mode section.