Suboptimal performance due to replication settings being set to 2 or 3

The file system write performance depends on the write performance of the storage volumes and its RAID configuration. However, in case the backend storage write performance is on par with its read performance, but the file system write performance is just 50% (half) or 33% (one-third) of the read performance, check if the file system replication is enabled.

Problem identification

When file system replication is enabled and set to 2, effective write performance becomes 50% of the raw write performance, since for every write operation, there are two internal write operation due to replication. Similarly, when file system replication is enabled and set to 3, effective write performance becomes approximately 33% of the raw write performance, since for every write operation, there are three internal write operation.

Issue the mmlsfs command and verify the default number of metadata and data replicas enabled on the file system. In the following sample output the metadata and data replication on the file system is set to 2:

# mmlsfs <fs> | grep replica | grep -i default


 -m                 2                        Default number of metadata replicas
 -r                 2                       Default number of data replicas

Issue the mmlsattr command to check whether replication is enabled at file level

# mmlsattr -L largefile.foo | grep replication


 metadata replication: 2 max 2
data replication:     2 max 2

Problem resolution and verification

The GPFS placement policy can be enforced to set the replication factor of temporary files for non-critical datasets to one. For example, temporary files like log files that can be recreated if necessary.

Follow these steps to set the replication value for log files to 1:

Create a placement_policy.txt file by using the following rule:

rule 'non-replicate-log-files' SET POOL 'SNCdata' REPLICATE (1) where lower(NAME) like '%.log'
rule 'default' SET POOL 'SNCdata'

Install the placement policy on the file system by using the following command:
```
mmchpolicy <fs> placement_policy.txt
```
Note: You can test the placement policy before installing it by using the following command:
```
mmchpolicy <fs> placement_policy.txt -I test
```
Issue one of the following commands to remount the file system for the policy to take effect:
Remount the file system on all the nodes by using one of the following commands:
- mmumount <fs> -N all
- mmmount <fs> -N all

Issue the mmlspolicy <fs> -L command to verify whether the output is as shown:

rule 'non-replicate-log-files' SET POOL 'SNCdata' REPLICATE (1) where lower(NAME) like '%.log'
rule 'default' SET POOL 'SNCdata'