mmqos command

Availability

Available on all IBM Storage Scale editions.

Description

The mmqos command is used to configure QoS in IBM Storage Scale to regulate I/O access to disk storage pools. This capability prevents I/O intensive processes from dominating storage access and adversely affecting other processes. Disk storage pools include the system storage pool and storage pools that the user defines.

For detailed information about QoS concepts, objects, and workflows, see Quality of service.

The mmqos command supports management of QoS user classes and capabilities such as dynamic QoS rebalancing (DQR) and limits for file system pools, in addition to providing all functionality previously available through the mmchqos and mmlsqos commands.

Parameters

class

Commands for working with QoS classes.

Note: The class create, class update, class updatekey, and class delete commands apply only to QoS user classes, not to QoS system classes. You cannot create, update, do an update key on, or delete a QoS system class.

create

Creates a QoS user class.

Device

The device name of the file system.

--class ClassName

The name of the user class.

[--fileset FilesetName [,FilesetName...]]

The names of zero or more filesets to be included in the user class. If you do not specify a fileset, you can add filesets to the class later with the class update add command.

update

Adds, replaces, or removes filesets from a QoS user class.

Device

The device name of the file system.

--class ClassName

The name of the QoS user class to be updated.

{--add | --replace | --remove}

The action to be done.

--add

Adds one or more filesets to the class.

--replace

Removes all filesets from the class and adds the specified filesets to the class. If a class has ten filesets and you specify --replace with two filesets, the command removes the ten filesets and adds the two filesets that you specify.

--remove

Removes a fileset from the class. You can remove only one fileset at a time.

Note: If the class includes only one fileset, you cannot remove it. If you want to keep the class but delete the fileset, you can add a second fileset to the class and then remove the first fileset. If you do not want to keep the class, you can delete the class with the class delete command.

[--fileset FilesetName [,FilesetName...]]

The names of one or more filesets to be added to or removed from the QoS user class. If the command is remove, you can specify only one fileset per command invocation.

delete

Deletes a QoS user class. If a class has throttles, you must delete the throttles before you can delete the class.

Device

The device name of the file system.

--class ClassName

The user class to be deleted.

set

Sets the value of an attribute for a QoS user class and system class.

Note: Other QoS attributes exist and are set at the file system level for all QoS user classes in the file system. For more information, see the description of the mmqos config set command.

Device

The device name of the file system.

--class ClassName

The user class or system class for which the attributes are set.

Attribute=Value[, Attribute=Value...]

The following attribute can be set:

dqr-enabled={yes | no}

Enables or disables DQR for the class. By default, the value of this class attribute for user classes shadows the value of the file-system-level dqr-enabled attribute. When you set the value of this class attribute here, the value becomes fixed and no longer shadows the value of the file-system-level attribute.

By default, DQR is enabled for QoS user classes and disabled for standard system classes.

list

Lists the QoS system classes and QoS user classes that are defined in the specified file system. For user classes, the command also lists the filesets that belong to the class.

Device

The device name of the file system.

[--config]

Instead of listing the filesets for each user class, the command lists the configuration settings for each user class. For more information, see the description of the mmqos class set option.

-Y

Displays the command output in a parseable format with a colon (:) as a field delimiter. Each column is described by a header.

Note: Fields that have a colon (:) are encoded to prevent confusion. For the set of characters that might be encoded, see the command documentation of mmclidecode. Use the mmclidecode command to decode the field.

throttle

Commands for working with QoS throttles. These commands apply to both system classes and user classes.

For more information, see Throttles for QoS classes.

create

Creates a QoS throttle for the specified file system, storage pool, and QoS class.

Device

The device name of the file system.

--pool [PoolName | default]

A storage pool.

--class ClassName

A QoS user class or a QoS system class.

-N {Node | NodeClass }

The local nodes among which the IOPS or MB/s for this throttle are to be divided. Also referred to as the throttle scope.

Node

The specified local node.

NodeClass

The nodes in the specified local node class. The node class mount is not supported.

For general information on how to specify node names, see Specifying nodes as input to GPFS commands.

-C {all | all_local | all_remote | ClusterName}

The nodes among which the IOPS or MB/s for this throttle are to be divided. Also referred to as the throttle scope.

all

All nodes that have mounted the file system, no matter which cluster the node belongs to. This group includes nodes in the cluster that owns the file system and nodes that belong to clusters that have remotely mounted the file system.

all_local

All nodes that have mounted the file system and that belong to the cluster that owns the file system.

all_remote

All nodes that have mounted the file system and are part of any cluster that has remotely mounted the file system.

ClusterName

All nodes that have mounted the file system and that belong to the specified cluster.

--maxiops MaxIOPS

The IOPS that is to be divided among the nodes governed by this throttle.

• The default value is unlimited.
• The valid range is 0 to 1,999,999,999.
• To specify a value less than 100, you must use the force option.

--maxmbs maxMBS

The megabytes per second (MB/s) of data transfer to be divided among the nodes governed by this throttle.

• The default value is unlimited.
• The valid range is 0 to 1,999,999,999.
• To specify a value less than 100, you must use the force option.

--force

Overrides the lower limit on the number of IOPS or MB/s that can be allocated to a throttle. QoS enforces a lower limit of 100 IOPS or 100 MB/s for the values set by using the --maxiops or --maxmbs option. To specify a value below these limits, you must use the --force option. Setting a value below the minimum may cause the processes affected by the throttle to run for an indefinitely long time.

update

Updates the IOPS limit or the MB/s limit of the specified throttle.

Important: If you specified the -N or -C option in the mmmqos throttle create command or if you used the -N or -C option in the mmqos throttle updatekey command to change the throttle scope, you must specify the same -N or -C option here as the current throttle scope. If you did not specify the -N or the -C option in either command, you do not have to specify it here because the default value is being used.

Device

The device name of the file system.

--pool [PoolName | default]

A storage pool. Be sure to specify the correct storage pool for the throttle that you want to update.

--class ClassName

A QoS user class or system class. Be sure to specify the correct class for the throttle that you want to update.

-N {Node | NodeClass }

The nodes among which the IOPS or MB/s for this throttle are to be divided. See the information about this option in the description of the mmqos throttle create command.

-C {all | all_local | all_remote | ClusterName}

The nodes among which the IOPS or MB/s for this throttle are to be divided. See the information about this option in the description of the mmqos throttle create command.

--maxiops maxIOPS

The IOPS limit for this throttle. See the information about this option in the description of the mmqos throttle create command.

--maxmbs maxMBS

The MB/s limit for this throttle. See the information about this option in the description of the mmqos throttle create command.

--force

Overrides the lower limit on the number of IOPS or MB/s that can be allocated to a throttle. See the information about this option in the description of the mmqos throttle create command.

Note:

• The four options such as --pool PoolName, --class ClassName, [{ -N {Node | NodeClass} | -C {all | all_remote | ClusterName} }] make up a unique key for throttle objects and this key cannot be updated. The throttle key must be deleted and can then be created again with new values.
• To change the attributes of the objects such as [--maxIOPS MaxIOPS] and [--maxMBS MaxMBS], provide the exact key values for which those values were set. Otherwise, you get the following error:

  mmqos: Processing the new QOS configuration... 
  mmqos: The QOS configuration record was not found. Nothing to update.

• If a key value is not set (because -N and -C are optional), it is still part of a key value but does not need to be supplied. That is, as these attributes are optional, if you did not specify them when creating the record, you should not specify them when updating it.

updatekey

Updates the throttle scope, which specifies the nodes across which the I/O limit of the throttle is to be applied. The throttle scope can be set by either the --new-N option or the --new-C option.

Important: If you specified the -N or -C option in the mmmqos throttle create command or if you used the -N or -C option in the mmqos throttle updatekey command to change the throttle scope, you must specify the same -N or -C option here as the current throttle scope. If you did not specify the -N or the -C option in either command, you do not have to specify it here because the default value is being used.

Device

The device name of the file system.

--pool [PoolName | default]

A storage pool. Be sure to specify the correct storage pool for the throttle that you want to update.

--class [ClassName | default]

A QoS user class or system class. Be sure to specify the correct class for the throttle that you want to update.

-N {Node | NodeClass }

The current throttle scope. See the information about this option in the description of the mmqos throttle create command.

-C {all | all_local | all_remote | ClusterName}

The current throttle scope. See the information about this option in the description of the mmqos throttle create command.

--new-N {Node | NodeClass }

The new throttle scope. See the information about the -N option in the description of the mmqos throttle create command.

--new-C {all | all_local | all_remote | ClusterName}

The new throttle scope. See the information about the -C option in the description of the mmqos throttle create command.

delete

Deletes the specified throttle.

Important: If you specified the -N or -C option in the mmmqos throttle create command or if you used the -N or -C option in the mmqos throttle updatekey command to change the throttle scope, you must specify the same -N or -C option here as the current throttle scope. If you did not specify the -N or the -C option in either command, you do not have to specify it here because the default value is being used.

Device

The device name of the file system.

--pool PoolName

The disk storage pool of the throttle that you want to delete.

--class ClassName

The QoS class for the throttle that you want to delete.

-N {Node | NodeClass }

The throttle scope of the throttle that you want to delete. See the information about this option in the description of the mmqos throttle create command.

-C {all | all_local | all_remote | ClusterName}

The throttle scope of the throttle that you want to delete. See the information about this option in the description of the mmqos throttle create command.

list

Lists the throttle system objects in the file system. You can specify either --pool or --class but not both. If you specify neither option, the command lists the throttle system objects for all pools and classes.

Device

The device name of the file system.

--pool PoolName

A storage pool. The command lists only the throttles that are configured to monitor the specified pool.

--class ClassName

A QoS class. The command lists only the throttles that are created for the specified class.

-Y

Displays the command output in a parseable format with a colon (:) as a field delimiter. Each column is described by a header.

Note: Fields that have a colon (:) are encoded to prevent confusion. For the set of characters that might be encoded, see the command documentation of mmclidecode. Use the mmclidecode command to decode the field.

config

Controls QoS attributes that apply at the file system level.

Note: To work with the same QoS attributes that are applicable at the class level, see the descriptions of the following commands:

• mmqos class set
• mmqos class list

set Device Attribute=Value[,Attribute=Value...]

Sets one or more file-system-level attributes to the specified values.

For example, the following command sets the fine-stats attribute to 30.

mmqos config set Device fine-stats=30

The following table shows the initial values of all the file-system-level attributes when QoS is initialized.

Table 1. Attributes at the file system level

Attribute	Value when QoS is initialized
fileset-stats	no
fine-stats	0
dqr-enabled	yes
pid-stats	no
skim-factor	0.987
stat-poll-interval	5
stat-slot-time	1000

Device

The device name of the file system.

Attribute=Value[,Attribute=Value...]

The name of the attribute and the value to set for the attribute. The following attributes can be set:

fileset-stats={yes | no}

Specifies whether QoS collects performance statistics at the fileset level for the class. The default value is no.

Fileset-level statistics are automatically enabled if a throttle is defined for any user class. This option is effective only when --fine-stats is nonzero and cannot be used with --pid-stats.

fine-stats=Seconds

Specifies how many seconds of fine-grained statistics QoS should retain in memory, allowing the mmqos report list command to display them. The default value is 0, which means that QoS does not save any fine-grained statistics. The valid range is 0 – 3840 seconds.

Fine-grained statistics are collected at every slat-slot-time interval and provide more detail than regular statistics. Their display is controlled by the --fine-stats-display-range option of the mmqos report list command.

For more information, see Analyzing fine-grained output from the mmqos report list command.

dqr-enabled={yes | no}

Enables or disables DQR.

By default, DQR is enabled for QoS user classes and disabled for standard system classes.

pid-stats= {yes | no}

Specifies whether QoS collects performance statistics for each process that accesses the storage pool through the class. The default value is no.

This option is effective only when --fine-stats is nonzero and is mutually exclusive with --fileset-stats.

skim-factor=Fraction

This attribute applies only to QoS system classes. If throttles from two or more QoS classes are operating and one class has a surplus of unused I/O capacity, QoS appropriates or "skims" the unused I/O capacity and assigns it to other classes that need more I/O capacity. The skim-factor attribute specifies the fraction of I/O capacity that cannot be skimmed from the class by other classes. The default value is 0.987, which means that only 1.3 percent of the I/O capacity of the class can be skimmed by other classes.

It is a good idea not to modify the value of this attribute without a strong reason.

stat-poll-interval=Seconds

Specifies the length of interval, in seconds, during which QoS statistics are collected by a node to be sent to the QoS manager node.

The valid range of values for stat-poll-interval is 5 - 60 seconds in one-second intervals. The stat-poll-interval must be a multiple of the stat-slot-time attribute.

stat-slot-time Milliseconds

Specifies the amount of time in milliseconds that QoS waits after the end of one stat-poll-interval before starting the next. The valid range is 100 to 60000. If the value is less than 1000 milliseconds, it must be one of the following values: 100, 125, 200, 250, or 500 milliseconds.

Both stat-poll-interval and stat-slot-time accept a special value of 0; when both are set to 0, the attributes revert to automatic adjustment mode.

For default values and guidance on selecting appropriate values based on the number of nodes mounting a file system, see QoS statistics and Scalability considerations.

list

Lists the current QoS attribute settings for the specified file system.

Note: You can issue this command action early in the configuration process to view the default attribute values. The following example shows the attribute after running the mmqos filesystem enable Device command, where Device is gpfs1:

# mmqos config list gpfs1
 Configuration Option            Configuration Value
----------------------------------------------------
 fine-stats                      0				
 pid-stats                       no
 stat-poll-interval              5
 stat-slot-time                  100
 fileset-stats                   no
 skim-factor                     0

All the settings that are listed in this example are default values.

Device

The device name of the file system.

-Y

Displays the command output in a parseable format with a colon (:) as a field delimiter. Each column is described by a header.

Note: Fields that have a colon (:) are encoded to prevent confusion. For the set of characters that might be encoded, see the command documentation of mmclidecode. Use the mmclidecode command to decode the field.

filesystem

Commands for controlling QoS processing for the specified file system.

init

Initializes the QoS environment for the specified file system. QoS initializes itself automatically if you issue a command that sets a configuration value, such as mmqos class create.

Device

The device name of the file system.

enable

Enables QoS processing for the specified file system. QoS begins or resumes regulating I/O operations and accumulating I/O performance statistics. This command does not modify any user-created QoS configuration and does not discard accumulated performance statistics.

Device

The device name of the file system.

disable

Disables QoS processing for the specified file system. QoS stops regulating I/O operations and ceases accumulating performance statistics. This command does not modify any user-created QoS configuration and does not discard accumulated performance statistics.

Device

The device name of the file system.

reset

Removes all user-created QoS configuration information and accumulated performance statistics for the specified file system. This operation resets the QoS configuration to the same state as initialized by the mmqos filesystem init command. This command fails if QoS is currently in the enabled state. You can use this option to delete and recover from an inconsistent or corrupted QoS configuration.

Warning: Issue this command only if you are certain that QoS functionality is no longer needed for the file system. No backup of the existing QoS configuration is created during this operation. If this feature is needed again after a reset, you must manually and entirely re-create the QoS configuration.

Before you run this command, make sure that any required QoS configuration information is separately backed up.

Device

The device name of the file system.

refresh

Refreshes the active QoS configuration in the IBM Storage Scale daemon from the main QoS configuration that is stored in the CCR.

Device

The device name of the file system.

list [-Y]

Lists the file systems in the current cluster that are configured for QoS. Notice that this command does not require a device name as the first option.

-Y

Displays the command output in a parseable format with a colon (:) as a field delimiter. Each column is described by a header.

Note: Fields that have a colon (:) are encoded to prevent confusion. For the set of characters that might be encoded, see the command documentation of mmclidecode. Use the mmclidecode command to decode the field.

status [-Y]

Displays the current status of QoS for the specified file system.

Device

The device name of the file system.

-Y

Displays the command output in a parseable format with a colon (:) as a field delimiter. Each column is described by a header.

Note: Fields that have a colon (:) are encoded to prevent confusion. For the set of characters that might be encoded, see the command documentation of mmclidecode. Use the mmclidecode command to decode the field.

The following example displays the QoS status for file system fs1:

# mmqos filesystem status fs1
                  Configuration Repository        Daemon
                  ------------------------        ------
State:            disabled                        disabled
Version:          49                              49
Throttling:       --                              inactive
Monitoring:       --                              inactive

The command displays QoS status information for the file system both as it exists in the CCR and as it exists in the file system:

State

Indicates whether QoS is initialized, enabled, disabled, or reset.

Version

Shows how many configuration changes have occurred since the first QoS configuration action was done for this file system.

Throttling

Indicates whether any throttle rules are created.

Note: In mmqos output the value "--" indicates that the measurement is not applicable in the current context. In the preceding example, it indicates that throttling and monitoring do not occur in the CCR environment.

Monitoring

Indicates whether performance statistics are being collected.

restore --restore-file RestoreFile

Restores the QoS configuration. Do not issue this command except with the guidance of IBM® Support for a disaster recovery for QoS configuration data.

Note: This function requires that the mmsdrbackup user exit for the CCR has been configured to ensure that there is a backup that includes the QoS data as part of the CCR backup.

report

Commands for reporting QoS statistics.

list

Writes a report to STDOUT.

Device

The device name of the file system.

--pool {all | PoolName}

The storage pools for which statistics are to be listed.

all

Lists statistics for all the pools that are assigned to QoS filesets.

PoolName

Lists statistics for the specified storage pool.

--seconds Seconds

Lists the QoS statistics that were collected during the previous specified number of seconds, relative to the current clock time. The valid range is from 1 to 6000 seconds; the default is 60 seconds. The values are reported over a series of subperiods that are determined by the specified number of seconds, as follows:

• When the value is less than or equal to 70, each subperiod duration is stat-poll-interval and it yields a maximum of approximately 70/stat-poll-interval subperiods.
• When the value is more than 70 and less than or equal to 720, the subperiod duration is 1 minute, with a maximum of 12 subperiods.
• When the value is more than 720, the subperiod duration is 10 minutes, with a maximum of 10 subperiods.

--sum-classes {yes | no}

yes

Lists the sum of the I/O performance values of all the QoS classes.

no

Lists the I/O performance of each QoS class separately. This value is the default.

--sum-nodes {yes |no}

yes

Lists the sum of the I/O performance values of all the nodes. This value is the default.

no

Lists the I/O performance of each node separately.

-Y

Displays the command output in a parseable format with a colon (:) as a field delimiter. Each column is described by a header.

Note: Fields that have a colon (:) are encoded to prevent confusion. For the set of characters that might be encoded, see the command documentation of mmclidecode. Use the mmclidecode command to decode the field.

--fine-stats-display-range FineStatsDisplayRange

Lists the fine-grained statistics that are currently available in memory. The FineStatsDisplayRange values specify the indexes of the first and last blocks of statistics in the range of blocks that you want to list, such as 0-4. If you specify only one integer, such as 2, the command lists all the blocks in memory beginning with that index and going to the end. The last line of the output lists the index of the next block in memory. You can avoid relisting fine-grained statistics by beginning the next report at this block index.

Fine-grained statistics are collected at stat-slot-time intervals and contain more information than regular statistics. They are intended to be used as input for programs that analyze and display data, such as the example plotting program at /usr/lpp/mmfs/samples/charts/qosplotfine.pl.

For more information, see Analyzing fine-grained output from the mmqos report list command and Analyzing regular output from the mmqos report list command.

Exit status

0

Successful completion.

Nonzero

A failure occurred.

Security

You must have root authority to run the mmqos command.

The node on which you enter the command must be able to run remote shell commands on any other administration node in the cluster. It must be able to do so without the use of a password and without producing any extraneous messages. For more information, see Requirements for administering a GPFS file system.