mmdeldisk command
Deletes disks from a GPFS™ file system.
Synopsis
mmdeldisk Device {"DiskName[;DiskName...]" | -F DescFile} [-a] [-c]
[-m | -r | -b] [-N {Node[,Node...] | NodeFile | NodeClass}]
[--inode-criteria CriteriaFile] [-o InodeResultFile]
[--qos QOSClass]
Availability
Available on all IBM Spectrum Scale™ editions.
Description
The mmdeldisk command migrates all data that would otherwise be lost to the remaining disks in the file system. It then removes the disks from the file system descriptor, preserves replication at all times, and optionally rebalances the file system after removing the disks.
- Copying unreplicated data off the disks and removing references to the disks (deldisk step).
- Rereplicating or rebalancing blocks across the remaining disks (restripe step).
Run the mmdeldisk command when system demand is low.
- If the file system is replicated, replica copies can be preserved at all times by using the default -r option or the -b option.
- Using the -m option will not preserve replication during the deldisk step because it will only copy the minimal amount of data off the disk being deleted so that every block has at least one copy. Also, using the -a option will not preserve replication during the deldisk step, but will then re-establish replication during the subsequent restripe step.
- If you want to move all data off the disk before running mmdeldisk, use mmchdisk to suspend all the disks that will be deleted and run mmrestripefs with the -r or -b option. This step is no longer necessary, now that mmdeldisk does the same function. If mmdeldisk fails (or is canceled), it leaves the disks in the suspended state, and mmdeldisk can be retried when the problem that caused mmdeldisk to stop is corrected.
- If the disk is permanently damaged and the file system is not replicated, or if the mmdeldisk command repeatedly fails, see the IBM Spectrum Scale: Problem Determination Guide and search for Disk media failure.
If the last disk in a storage pool is deleted, the storage pool is deleted. The mmdeldisk command is not permitted to delete the system storage pool. A storage pool must be empty in order for it to be deleted.
Results
- Data that has not been replicated from the target disks is migrated to other disks in the file system.
- Remaining disks are rebalanced, if specified.
Parameters
- Device
- The device name of the file system to delete the disks from. File system names need not be fully-qualified. fs0 is as acceptable as /dev/fs0. This must be the first parameter.
- "DiskName[;DiskName...]"
- Specifies the names of the disks to be deleted from the file system. If there is more than one disk to be deleted, delimit each name with a semicolon (;) and enclose the list in quotation marks.
- -F DiskFile
- Specifies a file that contains the names of the disks (one name per line), to be deleted from the GPFS cluster.
- -N {Node[,Node...] | NodeFile | NodeClass}
- Specifies the nodes that participate in the restripe of the file
system after the specified disks have been removed. This command supports
all defined node classes. The default is all or
the current value of the defaultHelperNodes parameter
of the mmchconfig command.
For general information on how to specify node names, see Specifying nodes as input to GPFS commands in the IBM Spectrum Scale: Administration and Programming Reference.
- --inode-criteria CriteriaFile
- Specifies the interesting inode criteria flag, where CriteriaFile is
one of the following:
- BROKEN
- Indicates that a file has a data block with all of its replicas
on disks that have been removed.Note: BROKEN is always included in the list of flags even if it is not specified.
- dataUpdateMiss
- Indicates that at least one data block was not updated successfully on all replicas.
- exposed
- Indicates an inode with an exposed risk; that is, the file has data where all replicas are on suspended disks. This could cause data to be lost if the suspended disks have failed or been removed.
- illCompressed
- Indicates an inode in which file compression or decompression is deferred, or in which a compressed file is partly decompressed to allow the file to be written into or memory-mapped.
- illPlaced
- Indicates an inode with some data blocks that might be stored in an incorrect storage pool.
- illReplicated
- Indicates that the file has a data block that does not meet the setting for the replica.
- metaUpdateMiss
- Indicates that there is at least one metadata block that has not been successfully updated to all replicas.
- unbalanced
- Indicates that the file has a data block that is not well balanced across all the disks in all failure groups.
Note: If a file matches any of the specified interesting flags, all of its interesting flags (even those not specified) will be displayed. - -o InodeResultFile
- Contains a list of the inodes that met the interesting inode flags
that were specified on the --inode-criteria parameter.
The output file contains the following:
- INODE_NUMBER
- This is the inode number.
- DISKADDR
- Specifies a dummy address for later tsfindinode use.
- SNAPSHOT_ID
- This is the snapshot ID.
- ISGLOBAL_SNAPSHOT
- Indicates whether or not the inode is in a global snapshot. Files in the live file system are considered to be in a global snapshot.
- INDEPENDENT_FSETID
- Indicates the independent fileset to which the inode belongs.
- MEMO (INODE_FLAGS FILE_TYPE [ERROR])
- Indicates the inode flag and file type that will be printed:Inode flags:
BROKEN exposed dataUpdateMiss illCompressed illPlaced illReplicated metaUpdateMiss unbalanced
File types:BLK_DEV CHAR_DEV DIRECTORY FIFO LINK LOGFILE REGULAR_FILE RESERVED SOCK *UNLINKED* *DELETED*
Notes:- An error message will be printed in the output file if an error is encountered when repairing the inode.
- DISKADDR, ISGLOBAL_SNAPSHOT, and FSET_ID work with the tsfindinode tool (/usr/lpp/mmfs/bin/tsfindinode) to find the file name for each inode. tsfindinode uses the output file to retrieve the file name for each interesting inode.
- --qos QOSClass
- Specifies the Quality of Service for I/O operations (QoS) class
to which the instance of the command is assigned. If you do not specify
this parameter, the instance of the command is assigned by default
to the maintenance QoS class. This parameter has
no effect unless the QoS service
is enabled. For more information, see the topic mmchqos
command in the IBM
Spectrum Scale: Administration
and Programming Reference. Specify
one of the following QoS classes:
- maintenance
- This QoS class is typically configured to have a smaller share of file system IOPS. Use this class for I/O-intensive, potentially long-running GPFS commands, so that they contribute less to reducing overall file system performance.
- other
- This QoS class is typically configured to have a larger share of file system IOPS. Use this class for administration commands that are not I/O-intensive.
Options
- -a
- Specifies that the mmdeldisk command not wait
for rereplicating or rebalancing to complete before returning. When
this flag is specified, the mmdeldisk command
runs asynchronously and returns after the file system descriptor is
updated and the rebalancing scan is started, but it does not wait
for rebalancing to finish. If no rebalancing is requested (-r option
is not specified), this option has no effect.
If -m is specified, this option has no effect. If -r or -b is specified (no option defaulting to -r), then the deldisk step is done using -m, and the restripe step is done using the specified option.
- -b
- Rebalances the blocks onto the other disks while moving data off
the disks being deleted. This might have to move much more data than
the -r operation. Note: Rebalancing of files is an I/O intensive and time consuming operation, and is important only for file systems with large files that are mostly invariant. In many cases, normal file update and creation will rebalance your file system over time, without the cost of the rebalancing.
- -c
- Specifies that processing continues even in the event that unreadable data exists on the disks being deleted. Data that has not been replicated is lost. Replicated data is not lost as long as the disks containing the replication are accessible.
- -m
- Does minimal data copying to preserve any data that is located
only on the disks being removed. This is the fastest way to get a
disk out of the system, but it could reduce replication of some blocks
of the files and metadata.Note: This might be I/O intensive if there is a lot of data to be copied or rereplicated off the disks that are being deleted.
- -r
- Preserves replication of all files and metadata during the mmdeldisk operation
(except when the -a option is specified).
This is the default.Note: This might be I/O intensive if there is a lot of data to be copied or rereplicated off the disks that are being deleted.
Exit status
- 0
- Successful completion.
- nonzero
- A failure has occurred.
Security
You must have root authority to run the mmdeldisk command.
The node on which the command is issued must be able to execute remote shell commands on any other node in the cluster without the use of a password and without producing any extraneous messages. For more information, see Requirements for administering a file system in IBM Spectrum Scale: Administration and Programming Reference.
Examples
- To delete gpfs1016nsd from file system fs1 and
rebalance the files across the remaining disks, issue this command:
The system displays information similar to:mmdeldisk fs1 gpfs1016nsd
Deleting disks ... Scanning sp1 storage pool Scanning user file metadata ... 100.00 % complete on Tue Mar 13 15:48:51 2012 Scan completed successfully. Checking Allocation Map for storage pool 'sp1' tsdeldisk64 completed. mmdeldisk: Propagating the cluster configuration data to all affected nodes. This is an asynchronous process.
- To
forcibly delete disk vmip2_nsd3, from file
system fs1, issue this command:
The system displays information similar to:mmdeldisk fs1 vmip2_nsd3 -p
Deleting disks ... Checking Allocation Map for storage pool system 68 % complete on Wed Apr 15 10:09:30 2015 100 % complete on Wed Apr 15 10:09:32 2015 Checking Allocation Map for storage pool data 73 % complete on Wed Apr 15 10:09:37 2015 100 % complete on Wed Apr 15 10:09:39 2015 GPFS: 6027-589 Scanning file system metadata, phase 1 ... GPFS: 6027-552 Scan completed successfully. GPFS: 6027-589 Scanning file system metadata, phase 2 ... Scanning file system metadata for data storage pool GPFS: 6027-552 Scan completed successfully. GPFS: 6027-589 Scanning file system metadata, phase 3 ... GPFS: 6027-552 Scan completed successfully. GPFS: 6027-589 Scanning file system metadata, phase 4 ... GPFS: 6027-552 Scan completed successfully. GPFS: 6027-565 Scanning user file metadata ... 100.00 % complete on Wed Apr 15 10:09:48 2015 \ (65792 inodes with total 397 MB data processed) GPFS: 6027-552 Scan completed successfully. GPFS: 6027-3908 Check file '/var/mmfs/tmp/fs1.pit.interestingInodes.12884901889' \ on vmip1 for inodes with broken disk addresses or failures. GPFS: 6027-000 Attention: A disk being removed reduces the number of failure groups to 1, which is below the number required for replication: 2. New blocks will be allocated from the remaining disks, but files will be unreplicated and hence at risk. GPFS: 6027-370 tsdeldisk completed. mmdeldisk: 6027-1371 Propagating the cluster configuration data to all affected nodes. This is an asynchronous process. #10:10:43# vmip1:~ # cat /var/mmfs/tmp/fs1.pit.interestingInodes.12884901889 This inode list was generated in the Parallel Inode Traverse on Wed Apr 15 10:09:47 2015 INODE_NUMBER DISKADDR SNAPSHOT_ID ISGLOBAL_SNAPSHOT FSET_ID MEMO(INODE_FLAGS FILE_TYPE [ERROR]) 57088 0:0 0 1 0 broken REGULAR_FILE Error: 226, Input/output error 57091 0:0 0 1 0 broken REGULAR_FILE Error: 226, Input/output error 57093 0:0 0 1 0 broken REGULAR_FILE Error: 226, Input/output error 57096 0:0 0 1 0 broken REGULAR_FILE Error: 226, Input/output error
- To identify the file names matching the "interesting" criteria
(in this case, having broken disk addresses), issue this command:
The system displays information similar to:tsfindinode -i /var/mmfs/tmp/fs1.pit.interestingInodes.12884901889 /fs1
This inode list was generated in the Parallel Inode Traverse on Wed Apr 15 10:11:47 2015 INODE_NUMBER DISKADDR SNAPSHOT_ID ISGLOBAL_SNAPSHOT FSET_ID MEMO(INODE_FLAGS FILE_TYPE [ERROR]) 57096 0 0xD4EB /fs1/file9 57088 0 0xC0CC /fs1/file1 57091 0 0x6788 /fs1/file4 57093 0 0x93F4 /fs1/file6