Restriping a GPFS file system

Writing data into a GPFS™ file system correctly stripes the file. However, if you have added disks to a GPFS file system that are seldom updated, use the mmrestripefs command to restripe the file system to achieve maximum performance. You can also use mmrestripefs to perform any incomplete or deferred file compression or decompression.

Restriping offers the opportunity to specify useful options in addition to rebalancing (-b option). Re-replicating (-r or -R option) provides for proper replication of all data and metadata. If you use replication, this option is useful to protect against additional failures after losing a disk. For example, if you use a replication factor of 2 and one of your disks fails, only a single copy of the data would remain. If another disk then failed before the first failed disk was replaced, some data might be lost. If you expect delays in replacing the failed disk, you could protect against data loss by suspending the failed disk using the mmchdisk command and re-replicating. This would assure that all data existed in two copies on operational disks.

If files are assigned to one storage pool, but with data in a different pool, the placement (-p) option will migrate their data to the correct pool. Such files are referred to as ill-placed. Utilities, such as the mmchattr command or policy engine, may change a file's storage pool assignment, but not move the data. The mmrestripefs command may then be invoked to migrate all of the data at once, rather than migrating each file individually. Note that the rebalance (-b) option also performs data placement on all files, whereas the placement (-p) option rebalances only the files that it moves.

If you do not replicate all of your files, the migrate (-m) option is useful to protect against data loss when you have an advance warning that a disk may be about to fail, for example, when the error logs show an excessive number of I/O errors on a disk. Suspending the disk and issuing the mmrestripefs command with the -m option is the quickest way to migrate only the data that would be lost if the disk failed.

If you do not use replication, the -m and -r options are equivalent; their behavior differs only on replicated files. After a successful re-replicate (-r option) all suspended disks are empty. A migrate operation, using the -m option, leaves data on a suspended disk as long as at least one other replica of the data remains on a disk that is not suspended. Restriping a file system includes re-replicating it; the -b option performs all the operations of the -m and -r options.

Use the -z option to perform any deferred or incomplete compression or decompression of files in the file system.

Consider the necessity of restriping and the current demands on the system. New data which is added to the file system is correctly striped. Restriping a large file system requires extensive data copying and may affect system performance. Plan to perform this task when system demand is low.

If you are sure you want to proceed with the restripe operation:

  1. Use the mmchdisk command to suspend any disks to which you do not want the file system restriped. You may want to exclude disks from file system restriping because they are failing. See Changing GPFS disk states and parameters.
  2. Use the mmlsdisk command to assure that all disk devices to which you do want the file system restriped are in the up/normal state. See Displaying GPFS disk states.
Specify the target file system with the mmrestripefs command. For example, to rebalance (-b option) file system fs1 after adding an additional RAID device, enter:
mmrestripefs fs1 -b
The system displays information similar to:
Scanning file system metadata, phase 1 ...
  19 % complete on Wed Mar 14 21:28:46 2012
 100 % complete on Wed Mar 14 21:28:48 2012
Scan completed successfully.
Scanning file system metadata, phase 2 ...
Scanning file system metadata for sp1 storage pool
Scan completed successfully.
Scanning file system metadata, phase 3 ...
Scan completed successfully.
Scanning file system metadata, phase 4 ...
Scan completed successfully.
Scanning user file metadata ...
 100.00 % complete on Wed Mar 14 21:28:55 2012
Scan completed successfully.
Note: Rebalancing of files is an I/O-intensive and time-consuming operation, and is important only for file systems with large files that are mostly invariant. In many cases, normal file update and creation will rebalance your file system over time, without the cost of the rebalancing.

For complete usage information, see mmrestripefs command.