Asynchronous auto prefetch
AFM can periodically fetch changed data asynchronously from a cloud object storage bucket to an AFM to cloud object storage fileset to synchronize the cache automatically. During the set prefetch interval, the created, modified, or deleted objects are synchronized to the cache. This synchronization helps AI applications with quick insights, reduced costs, improved performance, enhanced security, and streamlined operations.
Watch folder for the asynchronous auto prefetch
AFM supports all the clustered watch folder events to monitor filesets for file accesses events. With the clustered watch folder, you can monitor file operations across clusters by using a centralized tool that has scalability and resiliency built-in. The clustered watch folder captures file system activities, such as removal, deletion, or modification, at the IBM Storage Scale AFM fileset level and generates and streams notifications to topics in an external sink that you can manage. For more information, refer to Introduction to clustered watch folder.
You can enable an AFM to cloud object storage fileset for watch folder events to generate all the events fetched from the cloud object storage bucket to the cache. Watch folder events are generated on the cache when AFM downloads changed metadata and data to the cache.
For every created object in the cloud object storage target bucket, AFM fileset fetches and creates a file or directory with the same name on the cache. Similarly, after any object modification or object deletion, AFM synchronizes automatically to the cache. If a file is deleted from a cloud object storage bucket, then a fileset in the AFM RO, IW mode moves the deleted file in to the .ptrash directory under the AFM fileset.
Parameters for the asynchronous auto prefetch
- Use the --async-prefetch-interval parameter to set the interval. Valid values are in the range 0-2147483647. By default, this option is disabled. You can enable this option and set a 30-minute interval.
- Use the --fast-readdir2 parameter to enhance object synchronization, particularly for deleted objects. However, this parameter does not fetch or synchronize ACLs and xattrs.
- Use --async-prefetch-interval and --fast-readdir2 parameters during the fileset creation for new filesets.
- Use the following commands for the existing
filesets:
mmchfileset afmAsyncPrefetchInterval=<value>mmchfileset afmObjectFastReaddir2=yesNote: Both parameters must be enabled together for the asynchronous auto prefetch feature to function.