mmchdisk command

Availability

Available on all IBM Spectrum Scale™ editions.

Description

Use the mmchdisk command to change the state or the parameters of one or more disks in a GPFS file system.

The state of a disk is a combination of its status and availability, displayed with the mmlsdisk command. Disk status is normally either ready, emptied, suspended, or to be emptied. A transitional status such as replacing, replacement, to be emptied, suspended or being emptied might also appear if a disk is being deleted or replaced. An emptied disk indicates that there are no blocks allocated on the disk and neither will any blocks get allocated from the disk. Running the mmlsdisk command will show the disk status as emptied and will be removed faster without the metadata scan. A suspended or being emptied disk is one that the user has decided not to place any new data on. Existing data on a suspended or being emptied disk may still be read or updated. Typically, a disk is suspended prior to restriping a file system. Suspending a disk tells the mmrestripefs command that data is to be migrated off that disk. Disk availability is either up or down.

Be sure to use stop before you take a disk offline for maintenance. You should also use stop when a disk has become temporarily inaccessible due to a disk failure that is repairable without loss of data on that disk (for example, an adapter failure or a failure of the disk electronics).

The Disk Usage (dataAndMetadata, dataOnly, metadataOnly, or descOnly) and Failure Group parameters of a disk are adjusted with the change option.

See Recoverability considerations for details. The mmchdisk change command does not move data or metadata that resides on the disk. After changing disk parameters, in particular, Disk Usage, you may have to issue the mmrestripefs command with the -r option to relocate data so that it conforms to the new disk parameters.

The mmchdisk command can be issued for a mounted or unmounted file system. When maintenance is complete or the failure has been repaired, use the mmchdisk command with the start option. If the failure cannot be repaired without loss of data, you can use the mmdeldisk command.

DiskName:::DiskUsage:FailureGroup:::

Parameters

Device

The device name of the file system to which the disks belong. File system names need not be fully-qualified. fs0 is as acceptable as /dev/fs0.

This must be the first parameter.

suspend

or

empty

Instructs GPFS to stop allocating space on the specified disk. Place a disk in this state when you are preparing to restripe the file system off this disk because of faulty performance. This is a user-initiated state that GPFS never uses without an explicit command to change disk state. Existing data on a suspended disk may still be read or updated.

A disk remains in a suspended or to be emptied state until it is explicitly resumed. Restarting GPFS or rebooting nodes does not restore normal access to a suspended disk.

resume

Informs GPFS that a disk previously suspended is now available for allocating new space. If the disk is currently in a stopped state, it remains stopped until you specify the start option. Otherwise, normal read and write access to the disk resumes.

stop

Instructs GPFS to stop any attempts to access the specified disks. Use this option to tell the file system manager that a disk has failed or is currently inaccessible because of maintenance.

A disk remains stopped until it is explicitly started by the mmchdisk command with the start option.

You cannot run mmchdisk stop on a file system with replication 1.

start

Informs GPFS that disks previously stopped are now accessible. This is accomplished by first changing the disk availability from down to recovering. The file system metadata is then scanned and any missing updates (replicated data that was changed while the disk was down) are repaired. If this operation is successful, the availability is then changed to up. If the metadata scan fails, availability is set to unrecovered. This could occur if too many other disks are down. The metadata scan can be re-initiated at a later time by issuing the mmchdisk start command again.

If more than one disk in the file system is down, they must all be started at the same time by issuing the mmchdisk Device start -a command. If you start them separately and metadata is stored on any disk that remains down, the mmchdisk start command fails.

change

Instructs GPFS to change the disk usage parameter, the failure group parameter, or both, according to the values specified in the NSD stanzas.

-d "DiskDesc[;DiskDesc...]"

A descriptor for each disk to be changed.

Specify only disk names when using the suspend, resume, stop, or start options. Delimit multiple disk names with semicolons and enclose the list in quotation marks. For example, "gpfs1nsd;gpfs2nsd"

When using the change option, include the disk name and any new Disk Usage and Failure Group positional parameter values in the descriptor. Delimit descriptors with semicolons and enclose the list in quotation marks; for example, "gpfs1nsd:::dataOnly;gpfs2nsd:::metadataOnly:12".

The use of disk descriptors is discouraged.

-F StanzaFile

Specifies a file containing the NSD stanzas for the disks to be changed. NSD stanzas have this format:

%nsd: 
  nsd=NsdName
  usage={dataOnly | metadataOnly | dataAndMetadata | descOnly}
  failureGroup=FailureGroup
  pool=StoragePool
  servers=ServerList
  device=DiskName

where:

nsd=NsdName

The name of the NSD to change. For a list of disks that belong to a particular file system, issue the mmlsnsd -f Device, mmlsfs Device -d, or mmlsdisk Device command. The mmlsdisk Device command will also show the current disk usage and failure group values for each of the disks. This clause is mandatory for the mmchdisk command.

usage={dataOnly | metadataOnly | dataAndMetadata | descOnly}

Specifies the type of data to be stored on the disk:

dataAndMetadata

Indicates that the disk contains both data and metadata. This is the default for disks in the system pool.

dataOnly

Indicates that the disk contains data and does not contain metadata. This is the default for disks in storage pools other than the system pool.

metadataOnly

Indicates that the disk contains metadata and does not contain data.

descOnly

Indicates that the disk contains no data and no file metadata. Such a disk is used solely to keep a copy of the file system descriptor, and can be used as a third failure group in certain disaster-recovery configurations. For more information, see the IBM Spectrum Scale: Administration Guide and search on "Synchronous mirroring utilizing GPFS replication"

This clause is meaningful only for the mmchdisk change option.

failureGroup=FailureGroup

Identifies the failure group to which the disk belongs. A failure group identifier can be a simple integer or a topology vector that consists of up to three comma-separated integers. The default is -1, which indicates that the disk has no point of failure in common with any other disk.

GPFS uses this information during data and metadata placement to ensure that no two replicas of the same block can become unavailable due to a single failure. All disks that are attached to the same NSD server or adapter must be placed in the same failure group.

If the file system is configured with data replication, all storage pools must have two failure groups to maintain proper protection of the data. Similarly, if metadata replication is in effect, the system storage pool must have two failure groups.

Disks that belong to storage pools in which write affinity is enabled can use topology vectors to identify failure domains in a shared-nothing cluster. Disks that belong to traditional storage pools must use simple integers to specify the failure group.

pool=StoragePool

Specifies the storage pool to which the disk is to be assigned. If this name is not provided, the default is system.

Only the system storage pool can contain metadataOnly, dataAndMetadata, or descOnly disks. Disks in other storage pools must be dataOnly.

servers=ServerList

A comma-separated list of NSD server nodes. This clause is ignored by the mmadddisk command.

device=DiskName

The block device name of the underlying disk device. This clause is ignored by the mmadddisk command.

-a

Specifies to change the state of all of the disks belonging to the file system, Device. This operand is valid only on the resume and start options.

-N {Node[,Node...] | NodeFile | NodeClass }

Specifies a list of nodes that should be used for making the requested disk changes. This command supports all defined node classes. The default is all or the current value of the defaultHelperNodes parameter of the mmchconfig command.

For general information on how to specify node names, see Specifying nodes as input to GPFS commands.

--inode-criteria CriteriaFile

Specifies the interesting inode criteria flag, where CriteriaFile is one of the following:

BROKEN

Indicates that a file has a data block with all of its replicas on disks that have been removed.

Note: BROKEN is always included in the list of flags even if it is not specified.

dataUpdateMiss

Indicates that at least one data block was not updated successfully on all replicas.

exposed

Indicates an inode with an exposed risk; that is, the file has data where all replicas are on suspended disks. This could cause data to be lost if the suspended disks have failed or been removed.

illCompressed

Indicates an inode in which file compression or decompression is deferred, or in which a compressed file is partly decompressed to allow the file to be written into or memory-mapped.

illPlaced

Indicates an inode with some data blocks that might be stored in an incorrect storage pool.

illReplicated

Indicates that the file has a data block that does not meet the setting for the replica.

metaUpdateMiss

Indicates that there is at least one metadata block that has not been successfully updated to all replicas.

unbalanced

Indicates that the file has a data block that is not well balanced across all the disks in all failure groups.

Note: If a file matches any of the specified interesting flags, all of its interesting flags (even those not specified) will be displayed.

-o InodeResultFile

Contains a list of the inodes that met the interesting inode flags that were specified on the --inode-criteria parameter. The output file contains the following:

INODE_NUMBER

This is the inode number.

DISKADDR

Specifies a dummy address for later tsfindinode use.

SNAPSHOT_ID

This is the snapshot ID.

ISGLOBAL_SNAPSHOT

Indicates whether or not the inode is in a global snapshot. Files in the live file system are considered to be in a global snapshot.

INDEPENDENT_FSETID

Indicates the independent fileset to which the inode belongs.

MEMO (INODE_FLAGS FILE_TYPE [ERROR])

Indicates the inode flag and file type that will be printed:

Inode flags:

BROKEN
exposed
dataUpdateMiss
illCompressed
illPlaced      
illReplicated
metaUpdateMiss  
unbalanced 

File types:

BLK_DEV 
CHAR_DEV 
DIRECTORY
FIFO
LINK
LOGFILE
REGULAR_FILE
RESERVED
SOCK
*UNLINKED*
*DELETED*

Notes:

1. An error message will be printed in the output file if an error is encountered when repairing the inode.
2. DISKADDR, ISGLOBAL_SNAPSHOT, and FSET_ID work with the tsfindinode tool (/usr/lpp/mmfs/bin/tsfindinode) to find the file name for each inode. tsfindinode uses the output file to retrieve the file name for each interesting inode.

--qos QOSClass

Specifies the Quality of Service for I/O operations (QoS) class to which the instance of the command is assigned. If you do not specify this parameter, the instance of the command is assigned by default to the other QoS class. (Unlike other commands that have the --qos option, the mmchdisk command runs in the other class by default.) This parameter has no effect unless the QoS service is enabled. For more information, see the topic mmchqos command. Specify one of the following QoS classes:

maintenance

This QoS class is typically configured to have a smaller share of file system IOPS. Use this class for I/O-intensive, potentially long-running GPFS commands, so that they contribute less to reducing overall file system performance.

other

This QoS class is typically configured to have a larger share of file system IOPS. Use this class for administration commands that are not I/O-intensive.

For more information, see the topic Setting the Quality of Service for I/O operations (QoS).

Exit status

0

Successful completion.

nonzero

A failure has occurred.

Security

You must have root authority to run the mmchdisk command.

The node on which the command is issued must be able to execute remote shell commands on any other node in the cluster without the use of a password and without producing any extraneous messages. For more information, see Requirements for administering a GPFS file system.

See also

Location