mmrestripefs command

Availability

Available on all IBM Spectrum Scale™ editions.

Description

Run the mmrestripefs command to rebalance or restore the replication of all files in a file system. The command moves existing file system data between different disks in the file system based on changes to the disk state made by the mmchdisk, mmadddisk, and mmdeldisk commands. It also attempts to restore the metadata or data replication of all the files in the file system.

Alternatively, you can run the mmrestripefs command to perform any deferred or incomplete file compression or decompression in all the files of a file system.

You must specify one of the options (-m, -r, -b, -R, -c, -p, or -z) to indicate how much file system data to move or whether to perform file compression or decompression. You can run this command against a mounted or unmounted file system.

If the file system uses replication, then restriping the file system also replicates it. Also, if the file system uses replication the -r option and the -m options treat suspended disks differently. The -r option removes all data from a suspended disk. But the -m option leaves data on a suspended disk if at least one replica of the data remains on a disk that is not suspended.

The -b option performs all the operations of the -m and -r options.

Use the -z option to perform any deferred or incomplete file compression or decompression.

Consider the necessity of restriping and the current demands on the system. New data that is added to the file system is correctly striped. Restriping a large file system requires many insert and delete operations and might affect system performance. Plan to perform this task when system demand is low.

Parameters

Device

The device name of the file system to be restriped. File system names need not be fully qualified.

Device must be the first parameter. It can take the following parameters:

-m

Migrates all critical data off of any suspended disk in this file system. Critical data is all data that would be lost if currently suspended disks were removed.

-r

Migrates all data off suspended disks. It also restores all replicated files in the file system to their designated degree of replication when a previous disk failure or removal of a disk makes some replica data inaccessible. Use this parameter either immediately after a disk failure to protect replicated data against a subsequent failure, or before you take a disk offline for maintenance to protect replicated data against failure of another disk during the maintenance process.

-b

Rebalances all files across all disks that are not suspended, even if they are stopped. Although blocks are allocated on a stopped disk, they are not written to a stopped disk, nor are reads allowed from a stopped disk, until that disk is started and replicated data is copied onto it. The mmrestripefs command rebalances and restripes the file system. Use this option to rebalance the file system after you add, change, or delete disks in a file system.

Note: Rebalancing of files is an I/O intensive and time-consuming operation, and is important only for file systems with large files that are mostly invariant. In many cases, normal file update and creation will rebalance your file system over time, without the cost of the rebalancing.

-R

Changes the replication settings of each file, directory, and system metadata object so that they match the default file system settings (see the mmchfs command -m and -r options) as long as the maximum (-M and -R) settings for the object allow it. Next, it replicates or unreplicates the object as needed to match the new settings. This option can be used to replicate all of the existing files that were not previously replicated or to unreplicate the files if replication is no longer needed or wanted.

-c

Scans the file system and compares replicas of metadata and data for conflicts. When conflicts are found, the -c option attempts to fix the replicas.

--read-only

Modifies the -c option so that it does not try to fix conflicting replicas. You can use this option only with the -c option.

--metadata-only

Limits the specified operation to metadata blocks. Data blocks are not affected. This option is valid only with the -r, -b, -R, or-c option.

mmrestripefs command with this option completes its operation quicker than a full restripe, replication, or replica compare of data and metadata.

Use this option when you want to priortize the mmrestripefs operation on the metadata. This option ensures that the mmrestripefs operation has a reduced impact on the file system performance when compared to running the mmrestripefs command on the metadata and data.

After running the mmrestripefs command on the metadata with --metadata-only option, you can run the mmrestripefs command without this option to restripe the data and any metadata that requires to be restriped.

Note: This option does not run until all the nodes in the cluster are upgraded to IBM Spectrum Scale 4.2.1 release. If any of the nodes is not upgraded, the system displays the following error message:

mmrestripefs: The --metadata-only option support has not been enabled yet.
Run "mmchconfig release=LATEST" to activate the new function. 
mmrestripefs: Command failed. Examine previous error messages to determine cause.

-p

Directs mmrestripefs to repair the file placement within the storage pool.

Files that are assigned to one storage pool, but with data in a different pool, have their data migrated to the correct pool. Such files are referred to as ill-placed. Utilities, such as the mmchattr command, might change a file's storage pool assignment, but not move the data. The mmrestripefs command might then be invoked to migrate all of the data at once, rather than migrating each file individually. The placement option (-p) rebalances only the files that it moves. In contrast, the rebalance operation (-b) performs data placement on all files.

-z

Performs any deferred or incomplete file compression or decompression of files in the file system. For more information, see the topic File compression.

-P PoolName

Directs mmrestripefs to repair only files assigned to the specified storage pool. This option is convenient for migrating ill-placed data blocks between pools, for example after you change a file's storage pool assignment with mmchattr or mmapplypolicy with the -I defer flag.

Do not use for other tasks, in particular, for any tasks that require metadata processing, such as re-replication. By design, all GPFS™ metadata is kept in the system pool, even for files that have blocks in other storage pools. Therefore a command that must process all metadata must not be restricted to a specific storage pool.

-N {Node[,Node...] | NodeFile | NodeClass}

Specify the nodes that participate in the restripe of the file system. This command supports all defined node classes. The default is all or the current value of the defaultHelperNodes parameter of the mmchconfig command.

For general information on how to specify node names, see Specifying nodes as input to GPFS commands.

-o InodeResultFile

Contains a list of the inodes that met the interesting inode flags that were specified on the --inode-criteria parameter. The output file contains the following:

INODE_NUMBER

This is the inode number.

DISKADDR

Specifies a dummy address for later tsfindinode use.

SNAPSHOT_ID

This is the snapshot ID.

ISGLOBAL_SNAPSHOT

Indicates whether or not the inode is in a global snapshot. Files in the live file system are considered to be in a global snapshot.

INDEPENDENT_FSETID

Indicates the independent fileset to which the inode belongs.

MEMO (INODE_FLAGS FILE_TYPE [ERROR])

Indicates the inode flag and file type that will be printed:

Inode flags:

BROKEN
exposed
dataUpdateMiss
illCompressed
illPlaced      
illReplicated
metaUpdateMiss  
unbalanced 

File types:

BLK_DEV 
CHAR_DEV 
DIRECTORY
FIFO
LINK
LOGFILE
REGULAR_FILE
RESERVED
SOCK
*UNLINKED*
*DELETED*

Notes:

1. An error message will be printed in the output file if an error is encountered when repairing the inode.
2. DISKADDR, ISGLOBAL_SNAPSHOT, and FSET_ID work with the tsfindinode tool (/usr/lpp/mmfs/bin/tsfindinode) to find the file name for each inode. tsfindinode uses the output file to retrieve the file name for each interesting inode.

--inode-criteria CriteriaFile

Specifies the interesting inode criteria flag, where CriteriaFile is one of the following:

BROKEN

Indicates that a file has a data block with all of its replicas on disks that have been removed.

Note: BROKEN is always included in the list of flags even if it is not specified.

dataUpdateMiss

Indicates that at least one data block was not updated successfully on all replicas.

exposed

Indicates an inode with an exposed risk; that is, the file has data where all replicas are on suspended disks. This could cause data to be lost if the suspended disks have failed or been removed.

illCompressed

Indicates an inode in which file compression or decompression is deferred, or in which a compressed file is partly decompressed to allow the file to be written into or memory-mapped.

illPlaced

Indicates an inode with some data blocks that might be stored in an incorrect storage pool.

illReplicated

Indicates that the file has a data block that does not meet the setting for the replica.

metaUpdateMiss

Indicates that there is at least one metadata block that has not been successfully updated to all replicas.

unbalanced

Indicates that the file has a data block that is not well balanced across all the disks in all failure groups.

Note: If a file matches any of the specified interesting flags, all of its interesting flags (even those not specified) will be displayed.

--qos QOSClass

Specifies the Quality of Service for I/O operations (QoS) class to which the instance of the command is assigned. If you do not specify this parameter, the instance of the command is assigned by default to the maintenance QoS class. This parameter has no effect unless the QoS service is enabled. For more information, see the topic mmchqos command. Specify one of the following QoS classes:

maintenance

This QoS class is typically configured to have a smaller share of file system IOPS. Use this class for I/O-intensive, potentially long-running GPFS commands, so that they contribute less to reducing overall file system performance.

other

This QoS class is typically configured to have a larger share of file system IOPS. Use this class for administration commands that are not I/O-intensive.

For more information, see the topic Setting the Quality of Service for I/O operations (QoS).

Exit status

0

Successful completion.

nonzero

A failure has occurred.

Security

You must have root authority to run the mmrestripefs command.

The node on which you run the command must be able to execute remote shell commands on any other node in the cluster without the use of a password and without producing any extraneous messages. For more information, see Requirements for administering a GPFS file system.

See also

Location