AFM to cloud object storage support for more than 2 K metadata

AFM to cloud object storage now supports setting more than 2 K metadata on the fileset objects.

The S3 protocol has limitation of 2 K total object metadata. This limitation restricts true synchronization of metadata to and from cloud object storage where objects are set with more than 2 K metadata. AFM achieves this objective by splitting the extended metadata objects and keeping them as separate files in .afm directory that is created in the cloud object storage bucket. This method is useful when fileset is used for managing large numbers of users and groups or extended attributes and access controlled lists (ACLs) that is set through distributed file systems such as NFS.

When more than 2 K metadata (xattr and ACL size exceeds 1800 bytes. As 200 bytes are reserved for other AFM metadata attributes) is set on the object, its metadata is separated in the file that is stored in the .afm directory. The .afm directory is available on the cloud object storage, with the same path as the original object.

This feature is enabled by default on AFM to cloud object storage filesets.

Renaming an object with more than 2 K metadata deletes the old metadata object that is stored in .afm directory on cloud object storage. This process creates a new object that is kept with the renamed object name.

Similarly, deleting an object with more than 2 K metadata deletes the object in .afm directory on cloud object storage.

Note: Setting more than 2 K metadata on the AFM to cloud object storage fileset root (also known as the fileset path) is not supported.

Examples

mmafmctl fs1 getstate


Fileset Name    Fileset Target                              Cache State          Gateway Node    Queue Length   Queue numExec 
------------    --------------                               -------------        ------------    ------------   -------------    
fileset1        https://s3.amazonaws.com:443/fileset1        Active               c7f2n04         0              12

More than 2 K metadata is set on object object1 as shown in the following sample.

mmlsattr -L -d /gpfs/fs1/fileset1/object1


file name:            /gpfs/fs1/fileset1/object1
metadata replication: 1 max 2
data replication:     1 max 2
immutable:            no
appendOnly:           no
flags:                
storage pool name:    system
fileset name:         fileset1
snapshot name:        
creation time:        Fri Apr 29 03:19:52 2022
Misc attributes:      ARCHIVE
Encrypted:            no
gpfs.pcache.fname:    0x6F626A65637431
gpfs.pcache.attr:     0x00000000011000030000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
gpfs.pcache.inode:    0x0000000001100003A6A5783400000001
user.6EB9782EE4B058FB334: "58305B108681D6C6E49D3E93313F563AE7B42C4A3DB04872D000E942B710D4BCC60F25A52656ED9CE9CD21FE80E65932F3129B7B1D10ABC94967A55364ABDDDD1FFAA75C57D029973FFD551B9B778559453FD9C209233CB6BB4105CA0429973ED6EA0AC14F47B0D7B18C75679813041DBF8BA4CE315F236D5E9C304D9609A2751F1C4EA8FF713EF8C9509AD30ECB041242E90810892C7254BA4552851501A14E4375C954381AA6E603B564B79C93ED220A7C3D166210907A330A7B117A5577882F55D6C39E42EBC2FCC68D808D6F5E87EDDB49ED8206ED9DA6433C3C9ABF84661325D03534C2250CB40F03BAE796BFDD3F300664B26D86A4AA3BEF6C703300030340AA45CA341D8C8C77CE33E77E7717B5860CCC7F18DA557CC3B0798E7F6972F08FC4C44DD1826FF32A4A4C8BCF53153A40F153E2464B64F9E3330CECB40D948E47FABEC051BB5B44F77F36BFBBC4F434C2E70B75C312E56DDEC058003C43188DA74B2C1EF21E78C5696B16B52083801385EA2F912A092CF95546B0BD041953DD8B7BB1CDBC7E967EBB57C05D466A9372FEDB0A8B709071F2C4A85F5A5C4F0B20A0BAB462403B12ED786C7C7D9BD970A115CC92FE22A3591D28A9517EC5835357813242175BAA5C87521A3990E24459BDBF001176D213E9A3CBEF4803A48FE3DA3EA402BD4D027B63753BD3F78776B50E3B33DE11C243DB741B0EE0667ED33E1694A73B9FEE3BAB0B61A410B9E9C17D812285E25C098A496BA00C0B5F411CE9D1C66B8C88503A9954950A15966B255F615DDEA57F6AA04F4E5586B2C2DE82EE44C226796E41E9037590C438607E36D8AAAD12770C8E592DFBECF1552C4631EDB8BDA24BED937B177F3608C24088275E00BB3061EF55B6C25F95858D61589B79B24E2A00A566B390326B8F8E1E7A9E441834335D9B44FD52FCE8C77D5E8EDC2F5639A9635B848D78A506C7316CF4923F97830F75AF92D0A76BD22148D4D760607E37DBC8A6A775ED0C03D4CB892F98F17385E5D9C713D00DC1096D5F67EF0E0729C1FAAB1BEFBFD81E17B761E51F3266B37BD16FD5F85C16A3D5A378988CAE262910B808EF13B29657B0B284A005F3C2DE386314237E514B80B3347E5835CF89DB24E49171FD249B54F8966FF95B9D5787C6C5836808C941E3A8A079AD54AAB2279ED11392C6C08B323D3B7D8C127D595E820A5AC2082946C74F3BCC8EFFC4913144BAE7CF0F8454C94041CDFBCCF5D36272C5A95B8D0D2C77CC88974543B1624E404EA992790A671339CC10154D7BFB93CE267C9E699D0A034DF5E1B2E3EEE72B542D0BC75545F5832890A4397176992FE4CF0865EAAD0312CA37F70788E220115E0677B51632A447F8EFFF7885AB6BCBAC113D754A7EFC6F55E446CDB68224EC10E5D5B2020FF7"
Node1]

On the cloud object storage, it has object1 file in the bucket and its metadata that is stored in .afm/object1 object.

Do ls on bucket fileset1 by using command-line interface similar to following sample.

Node1] ls aws1/pillai1/


[2022-04-29 03:19:55 EDT]     0B object1
[2022-04-29 03:23:04 EDT]     0B .afm/

Node1] ls aws1/pillai1/.afm

[2022-04-29 03:19:55 EDT] 2.7KiB object1

When reading or downloading an object, it reads the extended object from the .afm directory as well and populates the right data and metadata.

Note: This mechanism uses extra storage space in addition to the storage used for an actual object data on cloud object storage.