mmchqos command

Changes the Quality of Service for I/O operations (QoS) settings for a file system.

Synopsis

mmchqos Device --enable [--reset] [--force]
   [--fine-stats Seconds] [--pid-stats {yes|no}]
   [[-N {Node[,Node...] | NodeFile | NodeClass}]
   [-C {all | all_remote | ClusterName[,ClusterName...]}]
   pool=PoolName[,QOSClass={nnnIOPS | unlimited}][,QOSClass=...] ...]

mmchqos Device --enable [--reset] [--force] -F StanzaFile
   [--fine-stats Seconds] [--pid-stats {yes|no}]

mmchqos Device --disable

Availability

Available on all IBM Spectrum Scale™ editions.

Description

With the mmchqos command, you can regulate I/O access to a specified storage pool by allocating shares of I/O operations to two QoS classes:

A maintenance class for I/O-intensive, potentially long-running GPFS™ commands. Typically you assign fewer IOPS to this class to prevent the I/O-intensive commands from dominating file system performance and significantly delaying other tasks.
An other class for all other processes. Typically you assign more IOPS or unlimited to this class so that normal processes have greater access to I/O resources and finish more quickly.

A third class, misc, is used to count the IOPS that some critical file system processes consume. You cannot assign IOPS to this class, but its count of IOPS is displayed in the output of the mmlsqos command.

When QoS is enabled, it restricts the active processes in a QoS class from collectively consuming more than the number of IOPS that you allocate to the class. It queues further I/O attempts until more I/O operations become available.

Remember the following points:

You can allocate shares of IOPS separately for each storage pool.
QoS divides each IOPS allocation equally among the specified nodes that have mounted the file system. See Table 1.
Allocations persist across unmounting and remounting the file system.
QoS stops applying allocations when you unmount the file system and resumes when you remount it.
When you change allocations or mount the file system, a brief delay due to reconfiguration occurs before QoS starts applying allocations.

For more information about this command, see Setting the Quality of Service for I/O operations (QoS).

Parameters

Device

The device name of the file system to which the command applies.

--enable

Causes QoS to start or to continue to apply IOPS allocations.

If you are specifying the --enable option for the first time, then QoS sets the QoS classes of pools that you do not specify in the command to unlimited IOPS.
On subsequent enables, QoS sets only the IOPS allocations that you specify in the command. It does not disturb other IOPS allocations. Compare the --reset parameter.

--disable

Causes QoS to stop applying IOPS allocations. Lets the file system run without any participation by QoS.

--reset

Causes QoS to set any QoS classes that you do not specify in the same command to unlimited IOPS.

When you enter multiple mmchqos commands for different storage pools, QoS typically records the settings for each pool and regulates the I/O consumption of each pool accordingly. However, with the --reset parameter, QoS discards the settings for all pools that are not specified in the same command and sets their QoS classes to unlimited. You can use this feature to reset the settings for any pools that you no longer want QoS to regulate and monitor.

--force

Causes QoS to accept an IOPS value lower than 100 IOPS.

Assigning less than 100 IOPS to a class is typically ineffective, because processes in that class run for an indefinitely long time. Therefore, the mmchqos command rejects IOPS values less than 100IOPS with an error message, unless you specify the --force option.

--fine-stats Seconds

Specifies how many seconds of fine-grained statistics to save in memory for the mmlsqos command to display. The default value is 0, which indicates that no fine-grained statistics are saved. The valid range is 0 - 3840 seconds.

Note: The value that you specify for Seconds is mapped to a larger actual value. The mmlsqos command reports the actual value.

Fine-grained statistics are taken at one-second intervals and contain more information than regular statistics. For more information, see the topic mmlsqos command.

--pid-stats {yes|no}

Enables or disables the keeping of fine-grained statistics for each QoS program that is active on each node of the cluster. The default value is no. This parameter is effective only if --fine-stats is not zero.

When this parameter is disabled, statistics reflect the combined data of all the QoS processes running on each node.

-N {Node[,Node...] | NodeFile | NodeClass}

Specifies nodes in the local cluster to which the IOPS are to be allocated. QoS divides the allocated IOPS equally among the specified nodes that have mounted the file system.

all: All the nodes in the cluster where the command is entered.
Node[,Node...]: The specified nodes.
NodeFile: A file that contains a list of nodes.
NodeClass: The nodes in the specified class.

For general information on how to specify node names, see Specifying nodes as input to GPFS commands.

This command does not support a NodeClass of mount.

-C {all | all_remote | ClusterName[,ClusterName...]}

Specifies clusters to which the IOPS are to be allocated. For each cluster, the IOPS are divided equally among the nodes that have mounted the file system.

all: All clusters, including both the local cluster (the cluster where the command is entered) and all remote clusters.
all_remote: All clusters other than the one where the command was entered.
ClusterName[,ClusterName...]: The specified clusters.

If neither the -N nor the -C parameter is specified, QoS divides the IOPS as described in the last row of Table 1.

Table 1. Allocation of IOPS
Option	Allocation of IOPS
-N {all \| Node[,Node...] \| NodeFile \| NodeClass}	The IOPS are divided equally among the specified nodes that have mounted the file system.
-C {all \| all_remote \| ClusterName[,ClusterName...]}	For each cluster, the IOPS are divided equally among the nodes that have mounted the file system.
Neither the -N nor the -C parameter is specified.	For the maintenance class: The IOPS are divided equally among all the nodes in the local cluster that have mounted the file system. other class: The IOPS are divided among all the nodes in the local cluster that have mounted the file system. Also, for each remote cluster, the IOPS are divided among all the nodes in the cluster that have mounted the file system.

pool=PoolName

Specifies a storage pool to whose maintenance or other class the IOPS are to be allocated. If you specify an asterisk (*) as the pool name, then the IOPS are allocated to the QoS classes of the unspecified pools. The unspecified pools are storage pools that you have not specified by name in any previous mmchqos command.

QOSClass={nnnIOPS | unlimited}

Identifies a QoS class and allocates IOPS to it.

QOSClass

The QoS class to which IOPS are assigned. You can specify one of the following classes:

maintenance: Most I/O-intensive, potentially slow-running GPFS administration commands run in this class by default. See the list of commands that support QoS in Table 2. Typically, you allocate fewer IOPS to this QoS class so that the commands that belong to it do not reduce overall file system performance.
When you start one of these commands, you can explicitly assign it to either QoS class. The assignment is effective only for the instance of the command that you are starting. In certain situations, you might assign one of these commands to the other class so that it runs faster and completes sooner.
other: All other processes that use I/O run in this class by default. Typically you assign more IOPS or unlimited to this class so that normal processes have greater access to I/O resources and finish more quickly.
Some I/O-intensive, potentially slow-running GPFS administration commands run in this class by default. (Currently just one: mmchdisk.) See the list of commands that support QoS in Table 2. When you start one of these commands, you can explicitly assign it to either QoS class. The assignment is effective only for the instance of the command that you are starting. In certain situations, you might want to assign one of these commands to the maintenance class so that normal processes can finish more quickly.

The following table lists the GPFS commands that support QoS and the QoS class that the command runs in by default:

Table 2. GPFS commands that support QoS
Commands that support QoS	Default QoS class
mmadddisk	`maintenance`
mmapplypolicy	`maintenance`
mmbackup	`maintenance`
mmchdisk	`other`
mmcheckquota	`maintenance`
mmdefragfs	`maintenance`
mmdeldisk	`maintenance`
mmdelfileset	`maintenance`
mmdelsnapshot	`maintenance`
mmdf	`maintenance`
mmfileid	`maintenance`
mmfsck	`maintenance`
mmimgbackup	`maintenance`
mmimgrestore	`maintenance`
mmlssnapshot	`maintenance`
mmrestripefs	`maintenance`
mmrpldisk	`maintenance`

nnnIOPS

You can use the following values for IOPS:

A value in the range 0IOPS - 1999999999IOPS. For an IOPS value less than 100, you must specify the -force option. Otherwise, QoS displays an error message like the following one:
```
maintenance=99iops is not reasonable. To insist, try --force.
```
QoS divides the IOPS allocation equally among the specified nodes that have mounted the file system.
You can type nnnIOPS either with or without the trailing characters IOPS. So either of the following two examples is valid:
```
(1) maintenance=400
(2) maintenance=400IOPS
```
unlimited: QoS does not restrict access to I/O operations.

-F StanzaFile

Specifies a file that contains QoS stanzas that describe allocations of IOPS. QoS stanzas have the following format:

%qos: 
  pool=PoolName
  class=QOSClass
  iops=Value
  nodes={Node[,Node...] | NodeFile | NodeClass}
  cluster={all | all_remote | ClusterName[,ClusterName...]}

where:

%qos

Identifies the stanza as a QoS stanza.

pool=PoolName

Specifies a storage pool to whose maintenance or other class the IOPS are allocated. If you specify an asterisk (*) as the pool name, then the IOPS are allocated to the QoS classes of unspecified pools. Unspecified pools are storage pools that you have not specified by name in any previous mmchqos command.

class=QOSClass

Specifies a QoS class. It must be maintenance or other.

iops=Value

Specifies the IOPS that are to be allocated to the QoS class. QoS divides the allocated IOPS among the specified nodes that have mounted the file system.

nodes={Node[,Node...] | NodeFile | NodeClass}

Specifies the nodes to which the IOPS are allocated.

For general information on how to specify node names, see Specifying nodes as input to GPFS commands.

This command does not support a NodeClass of mount.

cluster={all | all_remote | ClusterName[,ClusterName...]}

Specifies clusters to which the IOPS are to be allocated. For each cluster, the IOPS are divided equally among the nodes that have mounted the file system.

all: All clusters, including both the local cluster (the cluster where the command is entered) and all remote clusters.
all_remote: All clusters other than the one where the command was entered.
ClusterName[,ClusterName...]: The specified clusters.

Exit status

0: Successful completion.

Nonzero: A failure occurred.

Security

You must have root authority to run the mmchqos command.

The node on which you enter the command must be able to execute remote shell commands on any other administration node in the cluster. It must be able to do so without the use of a password and without producing any extraneous messages. For more information, see Requirements for administering a GPFS file system.

Examples

The following command enables QoS. If you are using the --enable option for the first time, then QoS sets the value of all the QoS classes to unlimited. If not, then the QoS classes retain their current settings:
```
mmchqos fs0 --enable
```
The following command disables QoS but does not change the current allocations of IOPS:
```
mmchqos fs0 --disable
```
The following command enables QoS and allocates 123 IOPS to the maintenance class of each unspecified pool:
```
mmchqos fs0 --enable pool=*,maintenance=123IOPS
```
You could also type the command like this:
```
mmchqos fs0 --enable pool=*,maintenance=123
```
The following command enables QoS and allocates 222 IOPS to the maintenance class of each unspecified pool. It also allocates 576 IOPS to the maintenance class of the pool mySSDs. You might make an allocation like this one to favor a pool of high-speed storage (mySSDs) that you expect to be accessed frequently. The command does not change the allocations of the QoS other classes:
Storage pool maintenance class other class

Each unspecified pool 222IOPS Unchanged

mySSDs 567IOPS Unchanged
```
mmchqos fs0 --enable pool=*,maintenance=222IOPS pool=mySSDs,maintenance=567IOPS
```
The following command enables QoS and allocates IOPS to the classes of the unspecified pools and three named pools:
Storage pool maintenance class other class

Each unspecified pool 444IOPS 555IOPS

system 111IOPS unlimited

second 222IOPS unlimited

third 333IOPS unlimited

The command is all on one line:
```
mmchqos fs0 --enable pool=*,maintenance=444IOPS,other=555iops
                pool=system,maintenance=111IOPS,other=unlimited
                pool=second,maintenance=222IOPS,other=unlimited
                pool=third,other=unlimited,maintenance=333IOPS
```
The following command enables QoS and allocates IOPS to the classes of three named pools:
Storage pool maintenance class other class

Each unspecified pool Unchanged Unchanged

system 111IOPS unlimited

second 222IOPS unlimited

third 333IOPS unlimited

The command is all on one line:
```
mmchqos fs0 --enable pool=system,maintenance=111IOPS,other=unlimited
                pool=second,maintenance=222IOPS,other=unlimited
                pool=third,other=unlimited,maintenance=333IOPS
```
The following command enables QoS and allocates IOPS to both classes of the system pool. Also, because the command contains the --reset parameter, it sets both classes of all the other storage pools in the file system to unlimited. The reset affects not only any unspecified pools, but also any named pools that are not explicitly mentioned in this command.
Storage pool maintenance class other class

system 111IOPS unlimited

Each unspecified pool, if any unlimited unlimited

Each named pool, if any, other than system unlimited unlimited

The command is all on one line:
```
mmchqos fs0 --enable --reset pool=system,maintenance=111IOPS,other=unlimited
```
The first part of the following command assigns IOPS to the QoS classes of the unspecified pools. It assigns 222 IOPS to the maintenance class of each unspecified pool.
The second part of the command allocates 456 IOPS to the other class of the storage pool mySAN, rather than allocating to it the typical value unlimited. You might make an allocation like this one to a SAN controller that serves both GPFS and other systems.

Storage pool maintenance class other class

Each unspecified pool 222IOPS unlimited

mySAN 123IOPS 456IOPS

The command is all on one line:
```
mmchqos fs0 --enable pool=*,maintenance=222IOPS
                pool=mySAN,other=456IOPS,maintenance=123IOPS
```
The following command allocates IOPS as they are specified in the stanza file qos.stanza:
```
mmchqos fs0 --enable -F /tmp/qos.stanza
```
The stanza file contains only one stanza:
```
%qos:
pool=sp1
class=maintenance
iops=800
nodes=node1,aixNodes
```
The command divides the allocated IOPS equally among the nodes node1 and the class aixNodes. Assuming that the node class contains three nodes, then 200 IOPS are assigned to each of the four nodes:
Storage pool or node maintenance class other class

sp1 and node1 200 IOPS Unchanged

Storage pool	maintenance class	other class
Each unspecified pool	222IOPS	Unchanged
`mySSDs`	567IOPS	Unchanged

Storage pool	maintenance class	other class
Each unspecified pool	444IOPS	555IOPS
`system`	111IOPS	unlimited
`second`	222IOPS	unlimited
`third`	333IOPS	unlimited

Storage pool	maintenance class	other class
Each unspecified pool	Unchanged	Unchanged
`system`	111IOPS	unlimited
`second`	222IOPS	unlimited
`third`	333IOPS	unlimited

Storage pool	maintenance class	other class
`system`	111IOPS	unlimited
Each unspecified pool, if any	unlimited	unlimited
Each named pool, if any, other than `system`	unlimited	unlimited

Storage pool	maintenance class	other class
Each unspecified pool	222IOPS	unlimited
`mySAN`	123IOPS	456IOPS

Storage pool or node	maintenance class	other class
`sp1` and `node1`	200 IOPS	Unchanged

Location

/usr/lpp/mmfs/bin