Configuration parameters for AFM, AFM-DR, and AFM to cloud object storage

Indicates the time when the requests start flushing to the home or the secondary cluster. This asynchronous delay is helpful for write-intensive applications that write to the same set of files. Because of this delay multiple writes to the home or the secondary cluster are replaced with a single write, which contains the latest data. However, if the parameter value is set high, the data updates on a remote cluster cause inconsistency.

afmAsyncOpWaitTimeout

Specifies the time when AFM waits for completion of any inflight asynchronous operation that is synchronizing with the home or the primary cluster. AFM cancels the asynchronous operation and synchronizes again after the home or the primary cluster is available.

Defines the frequency of revalidation that is triggered by a look-up operation such as ls or stat on a directory from the cache cluster. AFM sends a message to the home cluster to find out whether the metadata of that directory is modified since it was last revalidated. If so, the latest metadata information at the home cluster is reflected on the cache cluster.

Defines the frequency of revalidations that are triggered by the read and update operations on a directory from the cache cluster. AFM sends a message to the home cluster to find whether the metadata of that directory is modified since it was last revalidated.

Open requests on files or subdirectories on that directory are served from the cache fileset until the afmDirOpenRefreshInterval expires after which the open requests are sent to the home cluster.

AFM to cloud object storage supports directory objects. All directories with and without objects can now be synchronized to the cloud object storage. You can now set extended attributes on directories with the directory object support. To enable this afmObjectDirectoryObj parameter, stop the fileset, enable this parameter and start the fileset.

Defines the interval until which the AFM MDS or the primary cluster waits after it detects that the home or the secondary cluster is inaccessible before declaring the outage by moving the cache or the primary cluster state to Disconnected.

Indicates whether automatic eviction is triggered on a fileset.

If enabled on the cache or the primary cluster, exported paths from the home or the secondary cluster with Kerberos-enabled security levels like sys, krb5, krb5i, krb5p are mounted on the cache or the primary cluster in the increasing order of security level - sys, krb5, krb5i, krb5p. For example, the security level of exported path is krb5i then on the cache or the primary cluster, AFM tries to mount with level sys, followed by krb5, and finally mounts with the security level krb5i. If disabled on the cache or the primary cluster, exported paths from the home or the primary cluster are mounted with security level sys on the cache or primary cluster. You must configure KDC clients on all the gateway nodes on the cache or the primary cluster before you enable this parameter.

Is used with afmDisconnectTimeout to control the duration of a network outage between the cache and home clusters before the data in the cache expires and becomes unavailable until a home reconnection occurs.

Enable at the AFM cache and AFM-DR primary fileset level. AFM sends RPC to the gateway node for each update that is happening on the fileset. If the workload mostly involves new files creation, this parameter reduces the RPC exchanges between the application and the gateway node, improves the application performance, and minimizes the memory queue requirement at the gateway node.

Modifies the behavior that improves AFM performance when requests come for operations, such as lookup, open, readdir, or getattr, when AFM must fetch or read configuration of the fileset.

After you enable this parameter, AFM can store and access the AFM fileset configuration information from the last successful refresh operation directly from the cache. This optimization improves the performance of lookup, open or getattr calls for AFM.

If this parameter is not enabled, AFM must use the read-only locking mutex for the fileset configuration, which sometimes might lead to a locking contention. Hence, slowness is observed during lookup and readdir operations.

Defines the frequency of revalidation that is triggered by a look-up operation on a file such as ls or stat, from the cache cluster. AFM sends a message to the home cluster to determine that the metadata of the file is modified since it was last revalidated. If so, the latest metadata information at home is reflected on the cache cluster.

Defines the frequency of revalidations that are triggered by the read and write operations on a file from the cache cluster. AFM sends a message to the home cluster to determine that the metadata of the file is modified since it was last revalidated.

Open requests on the file are served from the cache fileset until the afmFileOpenRefreshInterval expires after which the open requests are sent to the home cluster.

Specifies the user-defined gateway node for an AFM or AFM DR fileset. When this parameter is set, it takes preference over the default gateway node that is assigned by AFM by using internal hashing algorithm. If the specified gateway node is not available, AFM internally assigns a gateway node from the available list by using the hashing algorithm.

This parameter value can be set to afmGateway = all for the AFM RO mode and the AFM LU mode during the file system-level migration. When this value is set, it improves the performance of data migration. This parameter with all value enforces AFM to migrate data by distributing migration task that is pertaining to inodes or files at the gateway node.

afmHashVersion=5 must be set at the cluster level by using the mmchconfig afmHashVersion=5 command.

IW, LU, RO, SW

(Value 'all' can be used for LU and RO.)

afmHardMemThreshold

Sets the maximum memory that AFM can use on each gateway node for handling queues. After this limit is reached, queues might not be handled on this gateway node due to lack of sufficient memory. Filesets belonging to this gateway node might go to a 'Dropped' state, depending on the activity. Exceeding the limit can occur if the cache cluster is disconnected for an extended time or if the connection with the home cluster has low bandwidth and therefore a lot of pending requests are accumulated in the queue. After the value of this parameter is changed, a gateway node daemon recycle is required for the new value to take effect.

Specifies the gateway node hashing algorithm that minimizes the impact of gateway nodes joining or leaving the active cluster by running as few recoveries as possible and balance mapping of AFM or AFM-DR filesets across the gateway node. Valid values are 1, 2, 4, and 5. Default value is 2. You can specify the value by using the mmchconfig command. For example, fit5, run the mmchconfig afmHashVersion=5 command.

This parameter can be enabled only when a home site is not an IBM Storage Scale or GPFS site.

When this parameter is set, AFM detects the snapshot path on the non-IBM Storage Scale or GPFS home during data migration and prefetch cases.

This map helps AFM to determine whether re validation of configuration is required by checking the latest cache entries populated in the map.

This parameter improves the performance and prevents any locking mutex contention that might be encountered in the cluster.

Specifies the number of filesets in the cluster on all file systems, on which recovery is run.

Specifies the mode in which the cache operates.

Changing from single-writer/read-only modes to read-only/local-updates/single-writer is supported. When changing from read-only to single-writer, the read-only cache is up-to-date. When changing from single-writer to read-only, all requests from cache should have been played at home. Changing from local-updates to read-only/local-updates/single-writer is restricted. A typical dataset is set up to include a single cache cluster in single-writer mode (which generates the data) and one or more cache clusters in local-updates or read-only mode. AFM single-writer/independent-writer filesets can be converted to primary. Primary/secondary filesets cannot be converted to AFM filesets.

Specifies the interval after which the primary gateway retries an operation on the home or the secondary cluster, in cases where the home or the secondary cluster is in an unhealthy state (see the Unmounted, Dropped states). Needs the GW node recycle or -i option for immediate effect.

When the afmMUAutoRemove option is set to 'yes' on MU mode fileset, remove operations on the files are queued on the gateway node to delete the files at cloud object storage automatically. After this option is set to yes, it overrides –from-cache and –from-target options of mmafmcosctl command.

Resets old AFM attributes on a disabled AFM fileset data and update the data with the new attributes. This option must be enabled, if any fileset is converted or promoted to an AFM MU mode fileset.

afmNFSVersion

Enables AFM NFSv4 supports.

Default value is 3 for compatibility with an earlier version. Allowed values are 3, 4.1 and 4.2 for Knfs protocol, and 3 and 4.1 for the NFS protocol.

afmNumFlushThreads

Defines the number of threads used on each gateway to synchronize updates to the home cluster. The default value is 4, which is sufficient for most installations. The current maximum value is 1024, which is too high for most installations. Ensure that you do not set this parameter to a very high value.

afmNumReadThreads

Defines the number of threads used on each participating gateway node during a parallel read. The default value of this parameter is 1. That is, one reader thread is active on every gateway node for each big read operation qualifying for splitting as per the parallel read threshold value.

afmNumWriteThreads

Defines the number of threads used on each participating gateway node during a parallel write. The default value of this parameter is 1. That is, one writer thread is active on every gateway node for each big write operation qualifying for splitting as per the parallel write threshold value.

afmObjectFastReaddir

Improves the objects download and readdir performance, when the afmObjectFastReaddir parameter value is set to 'yes' at the fileset level. Extended attributes and ACLs are not fetched from a cloud object storage when this parameter is enabled. Also, deleted objects on a cloud object storage system are not reflected immediately on a cache when this parameter is enabled. You can use this option to pull the objects into the cache to run quick analytics on the target data.

Specifies COS Key Expiration timeout value (in seconds). In case, the expiration timeout is set for access keys and secret keys at the Cloud Object Server, you can set expiration at the cache side for AFM to reload the access key and secret key values into the memory after the defined timeout. Initially keys are loaded into the memory first time when AFM Fileset is accessed.

After the expiration timeout is passed, AFM reload access and secret keys again into the memory and use the keys for communication purpose. You must update the access key and secret key once it is expired, before you start the next communication with server.

The afmObjKeyExpiration parameter is set at the cluster level. The valid values are 0 to 2147483647. The default is 36000.

When this parameter is enabled, the primary gateway node of a fileset at a cache cluster attempts to mount the exported path from multiple NFS servers that are defined in the mapping. Then, this primary gateway node sends unique messages through each NFS mount to improve performance by transferring data in parallel.

Before you enable this parameter, define the mapping between the primary gateway node and NFS servers by issuing the mmafmconfig command.

afmParallelReadChunkSize

Defines the minimum chunk size of the read that needs to be distributed among the gateway nodes during parallel reads. A zero (0) value disables the parallel reads across multiple gateways. The parallel reads are routed through a single gateway node.

afmParallelReadThreshold

Defines the threshold beyond which parallel reads become effective. Reads are split into chunks when the file size exceeds this threshold value. Values are in MB. The default value is 1024 MB.

afmParallelWriteChunkSize

Defines the minimum chunk size of the write that needs to be distributed among the gateway nodes during parallel writes. A zero (0) value disables the parallel writes across multiple gateways. The parallel writes are routed through a single gateway node.

afmParallelWriteThreshold

Defines the threshold beyond which parallel writes become effective. Writes are split into chunks when file size exceeds this threshold value. Values are in MB. The default value is 1024 MB.

Specifies the unique primary ID of the primary fileset for asynchronous data replication. This is used for connecting a secondary to a primary.

Enables AFM to perform one-time readdir of a directory from the home after the data migration to the cache and the application is started on the cache data. That is, the application is moved from the home to the cache and the application modifies the directory, which makes the directory dirty. When this parameter is set for a fileset, the prefetch operation is run on the fileset by using --readdir-only to move new or modified data from the home to the cache, even if the cache directory is dirty. After the data migration to the cache and the application is started on the cache data, this parameter synchronizes new files at the home directory to the cache for a single time.

When a sparse file at the home cluster is read into the cache, the cache cluster maintains the sparseness, if the size of the file exceeds the afmReadSparseThreshold. If the size of a file is less than the threshold, sparseness is not maintained at the cache cluster.

Cache data refresh operation in asynchronous mode improves performance of applications by querying data. Specify the value as 'yes' for the cache data refresh operation to be in asynchronous mode. In the IW mode, a revalidation request for files or directories is queued as an asynchronous request, and data in the last synchronized state is returned. The data is refreshed at cache after revalidation with home is complete. Revalidation time depends on the network availability and bandwidth.

Enables AFM to perform revalidation on files and directories only one time. This parameter improves the application performance after the data migration to the cache and the application is started on the cache data. When this parameter is set to yes, files and directories revalidation is performed only one time. Therefore, only one revalidation request goes to the home or target.

Enables optimized version of the recovery on the AFM Single Writer or AFM DR Primary fileset. After the optimized recovery version is set, AFM avoids using the older version of the recovery and uses the version V2 to identify the remove and rename operations. The new recovery version identify the home directory entries by running the GPFS policy at the home when recovery event is triggered.

afmResyncVer2

Increases the replication performance and scalability of a system that has heavy stress and workloads. It increases the message queuing performance by using an on-demand dependency resolution for queued messages, in case of recovery and/or resync is running.

Specifies the time that AFM waits for completion of revalidation to get response from the home cluster. Revalidation checks if any changes are available at home (data and metadata) that need to be updated to the cache cluster. Revalidation is performed when application trigger operations like lookup or open at cache. If revalidation is not completed within this time, AFM cancels the operation and returns data available at cache to the application.

Specifies the recovery point objective (RPO) time interval for an AFM DR fileset in minutes. Disabled is the default value of the parameter. You can also specify the value with the suffix M for minutes, H for hours, or W for weeks. For example, for 12 hours specify 12H. If you do not add a suffix, the value is assumed to be in minutes.

Specifies if the secondary is read-write or not.

Controls the visibility of the home snapshot directory in the cache cluster. For this to be visible in the cache cluster, this variable must be set to yes, and the snapshot directory name in the cache and home clusters must not be the same.

afmSyncNFSv4ACL

Enables the migration of NFSv4 ACLs from third-party file systems. when this parameter is enabled, data from a third-party storage is migrated to IBM Storage Scale.

Specifies the time that AFM waits for completion of any inflight synchronous operation that is synchronizing with the home or the primary cluster. When any application is performing any synchronous operation on the home or primary cluster, AFM tries to get a response from the home or the primary cluster. If the home or the primary cluster is not responding, application might be unresponsive. If operation does not complete in this timeout interval, AFM cancels the operation.

Creates a separate mount path at the gateway node through which synchronous read calls are sent to the home cluster separately. This arrangement helps other operations that are running on the AFM fileset to synchronize with the home cluster by using the default mount path.

If the home cluster is responding slow or busy, the synchronous read operations request that are coming through separate mount path at the gateway node gets killed and the read request is re-tried without affecting the other operations as the current operation is performed on the separate mount path.

In case of AFM (SW/IW) fileset , when you enable the afmSyncReadMount parameter, it avoids dropping the whole read request queue during a busy or slow home response, thus avoid sending EIO back to the application which is performing the read operation at the Cache cluster. This behavior skip moving AFM fileset into a stale state and avoid recovery process for AFM SW/IW fileset.

You can enable afmSyncReadMount parameter on the AFM Single-Writer (SW), Independent-Writer(IW), Local-Update(LU), and Read-Only(RO) mode fileset by using mmcrfileset or mmchfileset command.

The valid values for the afmSyncReadMount parameter are yes or no.

To set or unset the afmSyncReadMount parameter on an AFM Single-Writer or AFM Independent-Writer mode fileset, you must stop or unlink the fileset. For more information, see Stop and start replication on a fileset.

Identifies the home that is associated with the cache. The only allowed value is disable. It is used to convert AFM filesets to regular independent fileset.

mmchfileset fs1 ro -p afmTarget=disable

After an AFM fileset is converted to a regular fileset, the fileset cannot be changed back to an AFM fileset.