mmqos command

Availability

Available with IBM Spectrum Scale Advanced Edition, IBM Spectrum Scale Data Management Edition, IBM Spectrum Scale Developer Edition, or IBM Spectrum Scale Erasure Code Edition.

Description

You can assign I/O limits to QoS classes either as I/O operations per second (IOPS) or as megabytes per second (MB/s). You can display regular or fine-grained statistics of I/O accesses by processes over time.

mmchqos Device --reset --stat-poll-interval 0 --stat-slot-time 0
mmchqos Device --disable

For more information, see Setting the Quality of Service for I/O operations.

QoS user classes

QoS system classes

The QoS system classes provide file-system-wide allocation of IOPS or MB/s to four broad categories of processes. The disk storage pool can be the system storage pool or a user storage pool.

Creating and modifying throttles

QoS operation

• For each class, QoS divides the specified IOPS among the nodes in the throttle scope that have mounted the file system. If the allocation is static, QoS assigns an equal number of IOPS to each node and does not change the assignments. If the allocation is Mount Dynamic I/O, QoS periodically adjusts the assignment of IOPS to each node based on the relative frequency of I/O accesses to the disk storage pools of the class by the processes that are running on each node. For more information, see the topic Mount dynamic I/O (MDIO).

• If the class is a QoS user class, QoS regulates I/O accesses to files for which the following conditions are true:
  • The file is in a fileset that belongs to the specified user class.
  • The file data is located in the disk pool that is specified by the throttle.
  If the class is one of the QoS system classes maintenance or other, QoS regulates I/O accesses to files for which the following conditions are true:

  • The process that initiates the I/O access belongs to the specified QoS system class.
  • The file data is located in the disk pool that is specified by the throttle.

• A QoS component on each node monitors the I/O accesses of the processes that are running on the node. During each one-second period, for each QoS class, the QoS component allows a QoS I/O access if the IOPS or MB/s that are allocated to the node have not been exhausted during the current one-second period. Otherwise, QoS queues the I/O access until the next one-second period begins.

• When you change IOPS allocations, a brief delay due to reconfiguration occurs before QoS begins applying the new allocations.

• The following QoS operations apply to unmounting and mounting a file system:
  • QoS stops applying IOPS allocations to a file system when you unmount it and resumes when you remount it.
  • IOPS allocations persist across unmounting and remounting a file system.
  • When you mount a file system, a brief delay due to reconfiguration occurs before QoS starts applying allocations.

Mount dynamic I/O (MDIO)

QoS can dynamically balance the allocation of IOPS or MB/s among the nodes that have mounted the file system based on the relative proportions of attempted I/O accesses to the storage pool of the class by the processes running on each node. This feature is called MDIO. By default, MDIO is enabled for QoS user classes and is disabled for QoS system classes. You can enable or disable MDIO for a class with the mmqos class set command.

When MDIO is disabled for a class, QoS allocates the specified IOPS or MB/s of the class equally among all the nodes that have mounted the file system. QoS does not take into account the frequency of accesses to the storage pool of the class by the processes that are running on a node.

For example, suppose that MDIO is disabled in class CL01 and that the class is assigned 300 IOPS for storage pool P01. If three nodes A, B, and C have mounted the file system, QoS allocates 100 IOPS to each node for pool P01 and never changes the allocation. If the processes that are running on nodes B and C currently require only 50 IOPS of access to P01, their combined surplus of 100 IOPS goes unused. If node A requires a total of 200 IOPS of access to P01, it cannot draw the 100 IOPS that it needs from the unused IOPS on nodes B and C.

When MDIO is enabled for a class, QoS dynamically balances the allocation of IOPS or MB/s among the nodes that have mounted the file system. QoS determines the allocations based on the total number of attempted accesses to the storage pool of the class by processes on each node during an internally set time period. At the end of each time period, QoS calculates for each node the proportion of its attempted accesses to the total number of attempted accesses during the time period. If the proportions have changed since the end of the previous time period, QoS rebalances the allocations according to the new proportions.

For example, suppose that MDIO is now enabled in class C01 and that the class is again assigned 300 IOPS for storage pool P01. Again QoS initially assigns 100 IOPS to each of the nodes A, B, and C. However, QoS then monitors the number of attempted I/O accesses to pool P01 for class CL01 by the processes on each node. Suppose that during the most recent measurement period the number of attempted I/O accesses to P01 for CL01 were 240 from node A, 92 from node B, and 68 from node C, for a total of 400 attempted accesses. The proportions of the number of attempted accesses from each node compared to the total number of attempted accesses are .60 (240/400), .23 (92/400), and .17 (68/400). If these proportions are different from those of the previous time period, QoS rebalances the allocation of IOPS according to the new proportions. In this example 300 IOPS are specified for the class, so QoS allocates 180 (.60 x 300) IOPS to node A, 69 IOPS (.23 x 300) to node B, and 51 IOPS (.17 x 300) to node C.

Class sharing

Class sharing is an easy and powerful way to integrate throttles that use the same storage pool and to give them the ability to share surplus I/O accesses dynamically among themselves. To participate in class sharing, a class must be mdio-enabled. For more information, see the subtopic Mount dynamic I/O (MDIO).

For example, suppose that you have three existing throttles that regulate I/O accesses to the system pool and that have I/O limits of 200 IOPS, 300 IOPS, and 500 IOPS for a total of 1000 IOPS. You create a throttle for the mdio-all-sharing class and the system pool and assign an I/O limit of 1200 IOPS to the new throttle. Now the three individual throttles are collectively limited to 1200 IOPS. The individual throttles now have actual I/O limits of 240 IOPS (200/1000 x 1200), 360 IOPS (300/1000 x 1200), and 600 IOPS (500/1000 x 1200). If at some point in time the third throttle has a surplus of 200 IOPS, then 80 IOPS of the surplus (200/500 x 200) are dynamically assigned to the first throttle and 120 IOPS of the surplus (300/500 x 200) are dynamically assigned to the second throttle.

QoS statistics

On each node, a QoS component collects I/O statistics for each process that accesses a QoS disk storage pool. Periodically, the QoS component sends a report of the accumulated statistics to a QoS manager node. A QoS manager node stores and manages the statistics of QoS classes and nodes so that the statistics can be displayed by an mmqos report list command.

As the number of statistics reports increases, it can become so great that the QoS manager node is unable to process reports in a timely manner. If so, the statistics that are reported by the mmqos report list command might not reflect the actual number of I/O accesses.

Limitations

Parameters

class

Commands for working with QoS classes.

Note: The class create, class update, class updatekey, and class delete commands apply only to QoS user classes, not to QoS system classes. You cannot create, update, do an update key on, or delete a QoS system class.

create

Creates a QoS user class.

Device

The device name of the file system.

--class ClassName

The name of the user class.

[--fileset FilesetName [,FilesetName...]]

The names of zero or more filesets to be included in the user class. If you do not specify a fileset, you can add filesets to the class later with the class update add command.

update

Adds, replaces, or removes filesets from a QoS user class.

Device

The device name of the file system.

--class ClassName

The name of the QoS user class to be updated.

{--add | --replace | --remove}

The action to be done.

--add

Adds one or more filesets to the class.

--replace

Removes all filesets from the class and adds the specified filesets to the class. If a class has ten filesets and you specify --replace with two filesets, the command removes the ten filesets and adds the two filesets that you specify.

--remove

Removes a fileset from the class. You can remove only one fileset at a time.

Note: If the class includes only one fileset, you cannot remove it. If you want to keep the class but delete the fileset, you can add a second fileset to the class and then remove the first fileset. If you do not want to keep the class, you can delete the class with the class delete command.

[--fileset FilesetName [,FilesetName...]]

The names of one or more filesets to be added to or removed from the QoS user class. If the command is remove, you can specify only one fileset per command invocation.

delete

Deletes a QoS user class. If a class has throttles, you must delete the throttles before you can delete the class.

Device

The device name of the file system.

--class ClassName

The user class to be deleted.

set

Sets the value of an attribute for a QoS user class and system class.

Note: Other QoS attributes exist and are set at the file system level for all QoS user classes in the file system. For more information, see the description of the mmqos config set command.

Device

The device name of the file system.

--class ClassName

The user class or system class for which the attributes are set.

Attribute=Value[, Attribute=Value...]

The following attribute can be set:

mdio-enabled={yes | no}

Enables or disables MDIO for the class. By default, the value of this class attribute shadows the value of the file-system-level mdio-enabled attribute. When you set the value of this class attribute here, the value becomes fixed and no longer shadows the value of the file-system level attribute.

By default, MDIO is enabled for QoS user classes and is disabled for QoS system classes.

For more information, see the subtopic Mount dynamic I/O (MDIO) earlier in this topic.

list

Lists the QoS system classes and QoS user classes that are defined in the specified file system. For user classes, the command also lists the filesets that belong to the class.

Device

The device name of the file system.

[--config]

Instead of listing the filesets for each user class, the command lists the configuration settings for each user class. For more information, see the description of the mmqos class set option.

-Y

Displays the command output in a parseable format with a colon (:) as a field delimiter. Each column is described by a header.

Note: Fields that have a colon (:) are encoded to prevent confusion. For the set of characters that might be encoded, see the command documentation of mmclidecode. Use the mmclidecode command to decode the field.

throttle

Commands for working with QoS throttles. These commands apply to both system classes and user classes. For more information, see the subtopic Creating and modifying throttles earlier in this topic.

create

Creates a QoS throttle for the specified file system, storage pool, and QoS class.

Device

The device name of the file system.

--pool [PoolName | default]

A storage pool.

--class ClassName

A QoS user class or a QoS system class. For more information, see the subtopic QoS operation earlier in this topic.

-N {Node | NodeClass }

The nodes among which the IOPS or MB/s for this throttle are to be divided. Also referred to as the throttle scope. For more information, see the subtopic Creating and modifying throttles earlier in this topic.

Node

The specified node.

NodeClass

The nodes in the specified class. The node class mount is not supported.

For general information on how to specify node names, see Specifying nodes as input to GPFS commands.

-C {all | all_local | all_remote | ClusterName}

The nodes among which the IOPS or MB/s for this throttle are to be divided. Also referred to as the throttle scope. For more information, see the subtopic Creating and modifying throttles earlier in this topic.

all

All nodes that have mounted the file system, no matter which cluster the node belongs to. This group includes nodes in the cluster that owns the file system and nodes that belong to clusters that requested access to the file system.

all_local

All nodes that have mounted the file system and that belong to the cluster that owns the file system.

all_remote

All nodes that have mounted the file system and that belong to any cluster that requested access to the file system.

ClusterName

All nodes that have mounted the file system and that belong to the specified cluster.

--maxiops MaxIOPS

The IOPS that are to be divided among the nodes that are governed by this throttle. The default value is unlimited. The valid range is 0 - 1999999999. To specify a value less than 100, you must specify the force option. It is a good idea not to specify both an IOPS limit and an MB/s limit for the same throttle.

--maxmbs maxMBS

The megabytes per second of data transfer that are to be divided among the nodes that are governed by this throttle. The default value is unlimited. The valid range is 0 - 1999999999. To specify a value less than 100, you must specify the force option. It is a good idea not to specify both an IOPS limit and an MB/s limit for the same throttle.

--force

Overrides the lower limit on the number of IOPS or MB/s that can be allocated to a throttle. QoS enforces a lower limit of 100 IOPS or 100 MB/s on the value that the --maxiops option or the --maxmbs option can be set to. Setting a value below the limit with the --force option typically causes the processes that are affected by the throttle to run for an indefinitely long time.

update

Updates the IOPS limit or the MB/s limit of the specified throttle.

Important: If you specified the -N or -C option in the mmmqos throttle create command or if you used the -N or -C option in the mmqos throttle updatekey command to change the throttle scope, you must specify the same -N or -C option here as the current throttle scope. If you did not specify the -N or the -C option in either command, you do not have to specify it here, because the default value is being used. For more information, see the subtopic Creating and modifying throttles earlier in this topic.

Device

The device name of the file system.

--pool [PoolName | default]

A storage pool. Be sure to specify the correct storage pool for the throttle that you want to update. For more information, see the subtopic Creating and modifying throttles earlier in this topic.

--class ClassName

A QoS user class or system class. Be sure to specify the correct class for the throttle that you want to update. For more information, see the subtopic Creating and modifying throttles earlier in this topic.

-N {Node | NodeClass }

The nodes among which the IOPS or MB/s for this throttle are to be divided. See the information about this option in the description of the mmqos throttle create command.

-C {all | all_local | all_remote | ClusterName}

The nodes among which the IOPS or MB/s for this throttle are to be divided. See the information about this option in the description of the mmqos throttle create command.

--maxiops maxIOPS

The IOPS limit for this throttle. See the information about this option in the description of the mmqos throttle create command.

--maxmbs maxMBS

The MB/s limit for this throttle. See the information about this option in the description of the mmqos throttle create command.

--force

Overrides the lower limit on the number of IOPS or MB/s that can be allocated to a throttle. See the information about this option in the description of the mmqos throttle create command.

Note:

• The four options such as --pool PoolName, --class ClassName, [{ -N {Node | NodeClass} | -C {all | all_remote | ClusterName} }] make up a unique key for throttle objects and this key cannot be updated. The throttle key must be deleted and can then be created again with new values.
• To change the attributes of the objects such as [--maxIOPS MaxIOPS] and [--maxMBS MaxMBS], provide the exact key values for which those values were set. Otherwise, you will get the following error:

  mmqos: Processing the new QOS configuration... 
  mmqos: The QOS configuration record was not found. Nothing to update.

• If a key value is not set (because -N and -C are optional), it is still part of a key value but does not need to be supplied. That is, as these attributes are optional, if you did not specify them when creating the record, you should not specify them when updating it.

updatekey

Updates the throttle scope, which specifies the nodes across which the I/O limit of the throttle is to be applied. The throttle scope can be set by either the -new-N option or the -new-C option. For more information, see the subtopic Creating and modifying throttles earlier in this topic.

Important: If you specified the -N or -C option in the mmmqos throttle create command or if you used the -N or -C option in the mmqos throttle updatekey command to change the throttle scope, you must specify the same -N or -C option here as the current throttle scope. If you did not specify the -N or the -C option in either command, you do not have to specify it here, because the default value is being used. For more information, see the subtopic Creating and modifying throttles earlier in this topic.

Device

The device name of the file system.

--pool [PoolName | default]

A storage pool. Be sure to specify the correct storage pool for the throttle that you want to update. For more information, see the subtopic Creating and modifying throttles earlier in this topic.

--class [ClassName | default]

A QoS user class or system class. Be sure to specify the correct class for the throttle that you want to update. For more information, see the subtopic Creating and modifying throttles earlier in this topic.

-N {Node | NodeClass }

The current throttle scope. See the information about this option in the description of the mmqos throttle create command.

-C {all | all_local | all_remote | ClusterName}

The current throttle scope. See the information about this option in the description of the mmqos throttle create command.

-new-N {Node | NodeClass }

The new throttle scope. See the information about the -N option in the description of the mmqos throttle create command.

-new-C {all | all_local | all_remote | ClusterName}

The new throttle scope. See the information about the -C option in the description of the mmqos throttle create command.

delete

Deletes the specified throttle.

Important: If you specified the -N or -C option in the mmmqos throttle create command or if you used the -N or -C option in the mmqos throttle updatekey command to change the throttle scope, you must specify the same -N or -C option here as the current throttle scope. If you did not specify the -N or the -C option in either command, you do not have to specify it here, because the default value is being used. For more information, see the subtopic Creating and modifying throttles earlier in this topic.

Device

The device name of the file system.

--pool PoolName

The disk storage pool of the throttle that you want to delete. For more information, see the subtopic Creating and modifying throttles earlier in this topic.

--class ClassName

The QoS class for the throttle that you want to delete. For more information, see the subtopic Creating and modifying throttles earlier in this topic.

-N {Node | NodeClass }

The throttle scope of the throttle that you want to delete. See the information about this option in the description of the mmqos throttle create command.

-C {all | all_local | all_remote | ClusterName}

The throttle scope of the throttle that you want to delete. See the information about this option in the description of the mmqos throttle create command.

list

Lists the throttle system objects in the file system. You can specify either --pool or --class but not both. If you specify neither option, the command lists the throttle system objects for all pools and classes.

Device

The device name of the file system.

--pool PoolName

A storage pool. The command lists only the throttles that are configured to monitor the specified pool.

--class ClassName

A QoS class. The command lists only the throttles that are created for the specified class.

-Y

Displays the command output in a parseable format with a colon (:) as a field delimiter. Each column is described by a header.

Note: Fields that have a colon (:) are encoded to prevent confusion. For the set of characters that might be encoded, see the command documentation of mmclidecode. Use the mmclidecode command to decode the field.

config

Controls QoS attributes that apply at the file system level.

Note: To work with the same QoS attributes at the class level, see the descriptions of the following commands:

• mmqos class set
• mmqos class list

set Device Attribute=Value[,Attribute=Value...]

Sets the initial value to be assigned to an attribute of a class when the class is created. For example, the following command causes the class attribute fileset-stats to be initialized to yes in each subsequent new class. This command does not alter the value of fileset-stats in existing classes:

mmqos config set Device fileset-stats=yes

If the initial value is later set to no, as in the following command, then fileset-stats is initialized to no in each subsequent new class. This command does not affect the value of fileset-stats in existing classes:

mmqos config set Device fileset-stats=no

You cannot set the value yes for pid-stats and fileset-stats attributes at a time. That is, either pid-stats=yes or fileset-stats=yes can be used in the configuration.

The following table shows the initial values of all the file-system-level attributes when QoS is initialized:

Table 4. Attributes at the file system level

Attribute	Value when QoS is initialized
fileset-stats	no
fine-stats	0
mdio-enabled	yes
pid-stats	no
skim-factor	.987
stat-poll-interval	5
stat-slot-time	1000

Device

The device name of the file system.

Attribute=Value[,Attribute=Value...]

The name of the attribute and the value to set for the attribute. The following attributes can be set:

fileset-stats={yes | no}

Specifies whether QoS keeps statistics for each fileset that is associated with the class. The default value is no, which means that QoS does not keep statistics.

fine-stats=Seconds

Specifies how many seconds of fine-grained statistics to save in memory so that the mmqos report list command can list them. The default value is 0, which means that QoS does not save any fine-grained statistics. The valid range is 0 - 3840 seconds.

Fine-grained statistics are collected at specified intervals and contain more information than regular statistics. The interval at which fine-grained statistics are collected is set with the slat-slot-time file-system-level attribute. The display of fine-grained statistics is controlled by the --fine-stats-display-range option of the mmqos report list command.

mdio-enabled={yes | no}

Enables or disables MDIO. When MDIO is enabled for a class, QoS dynamically balances the allocation of IOPS or MB/s among the nodes that have mounted the file system, based on the relative proportions of their attempted I/O accesses to the storage pool of the class. When MDIO is disabled for a class, QoS allocates a fixed number of IOPS or MB/s among the nodes that have mounted the file system and never changes the allocation.

By default, MDIO is enabled for QoS user classes and is disabled for QoS system classes. You can change the value of this attribute for individual classes with the mmqos class set command.

For more information, see the subtopic Mount dynamic I/O (MDIO) earlier in this topic.

pid-stats= {yes | no}

Specifies whether QoS keeps statistics for each process that accesses the storage pool of a QoS class. The default value is no, which means that QoS does not keep statistics for each process.

skim-factor=Fraction

This attribute applies only to QoS system classes. If throttles from two or more QoS classes are operating and one class has a surplus of unused I/O capacity, QoS appropriates or "skims" the unused I/O capacity and assigns it to other classes that need more I/O capacity. The skim-factor attribute specifies the fraction of I/O capacity that cannot be skimmed from the class by other classes. The default value is .987, which means that only 1.3 percent of the I/O capacity of the class can be skimmed by other classes.

It is a good idea not to modify the value of this attribute without a strong reason.

stat-poll-interval=Seconds

Specifies the interval in seconds between each sending of accumulated QoS statistics by a node to the QoS manager node. The QoS manager node is an internally selected node that stores and manages the statistics of a QoS class so that the statistics can be displayed by an mmqos report list command. The QoS statistics are described in the following subtopics, which appear later in this topic:

• Analyzing regular output from the mmqos report list command
• Analyzing fine-grained output from the mmqos report list command

At the beginning of each stat-poll-interval, the QoS component on the node sets the node's QoS statistics variables to zero. During the stat-poll-interval interval, the QoS component adds an amount to the appropriate statistics variable whenever a relevant event occurs. For example, when a process on a node consumes some number of IOPS, the QoS component adds the IOPS count to the statistics variable for the number of IOPS consumed by the node. At the end of the interval, the QoS component on the node sends the accumulated value for QoS statistics to the QoS manager node.

The valid range of values for stat-poll-interval is 0 - 120 seconds in one-second intervals. The default value is 5 seconds, which means that a node collects QoS statistics for 5 seconds before it sends the accumulated QoS statistics to the QoS manager node. The stat-poll-interval must be a multiple of the value of the stat-slot-time attribute. For more information, see the description of that attribute.

Reducing the frequency of QoS statistics reports:

For any QoS class, the number of statistics reports that the QoS manager node must process depends on the following factors:

The number of nodes that mount the file system

As the number of nodes increases, the number of reports also increases.

The value of stat-poll-interval

As the value decreases, the frequency of reports increases. In other words, the shorter that the data collection interval is, the greater are the number of reports that are sent per second or per minute to the QoS manager node.

The value of stat-slot-time

The stat-slot-time attribute provides a way to reduce the frequency of statistics reports that are sent to the QoS manager node. The value of stat-slot-time sets the length of the delay that QoS interposes between the end of one stat-poll-interval and the beginning of the next. As the value of stat-slot-time increases, the frequency of reports decreases. In other words, the greater that the delay is between reporting intervals, the fewer are the reports that are sent per second or per minute to the QoS manager node. For more information, see the description of stat-slot-time attribute that appears immediately after the description of stat-slot-time in this topic.

If the flow of new reports becomes too great, the QoS manager node can no longer update its stored statistics values in a timely manner. In that case, the statistics that are reported by the mmqos report list command can be incorrect.

Two options are available to compensate for the increase in the number of reports that occurs when the number of nodes that mount the file system increases by a significant amount:

• Activate the dynamic adjustment of stat-poll-interval and stat-slot-time by setting both attributes to zero. To suspend dynamic adjustment, set the value of either attribute to a nonzero value. For more information, see the subtopic QoS statistics earlier in this topic.
• Manually increase the values of stat-poll-interval and stat-slot-time. For guidance in selecting values for stat-poll-interval and stat-slot-time, see Table 3.

stat-slot-time Milliseconds

Specifies the amount of time in milliseconds that QoS waits after the end of one stat-poll-interval before starting the next. Valid values are 125, 250, 500, 1000, and 2000 - 24000 in 1000 millisecond increments. The default value is 125. Increasing the value of stat-slot-time reduces the number of QoS statistics reports that are sent to the QoS manager node.

list

Lists the current settings of the QoS data collection attributes for this file system.

Note: You can issue this command action early in your configuration process to see default data collection attribute values that have already been set. The following example shows the settings after mmqos filesystem enable Device is issued, where Device is gpfs1:

# mmqos config list gpfs1
 Configuration Option            Configuration Value
----------------------------------------------------
 fine-stats                      0				
 pid-stats                       no
 stat-poll-interval              5
 stat-slot-time                  100
 fileset-stats                   no
 skim-factor                     0

All the settings that are listed in this example are default values.

Device

The device name of the file system.

-Y

Displays the command output in a parseable format with a colon (:) as a field delimiter. Each column is described by a header.

Note: Fields that have a colon (:) are encoded to prevent confusion. For the set of characters that might be encoded, see the command documentation of mmclidecode. Use the mmclidecode command to decode the field.

filesystem

Commands for controlling QoS processing for the specified file system.

init

Initializes the QoS environment for the specified file system. QoS initializes itself automatically if you issue a command that sets a configuration value, such as mmqos class create.

Device

The device name of the file system.

enable

Enables QoS processing for the specified file system. QoS begins or resumes regulating I/O operations and accumulating I/O performance statistics. This command does not modify any customer-created QoS configuration and does not discard accumulated performance statistics.

Device

The device name of the file system.

disable

Disables QoS processing for the specified file system. QoS stops regulating I/O operations and stops accumulating performance statistics. This command does not modify any customer-created QoS configuration and does not discard accumulated performance statistics.

Device

The device name of the file system.

reset

Removes all QoS customer-created QoS configuration information and accumulated performance statistics from the specified file system. Resets the QoS configuration to the same state that mmqos filesystem init sets. This command fails if QoS is in the enabled state.

Warning: Issue this command only if you are sure that you no longer need QoS functions in this file system. If you remove all the QoS configuration information and later find that you need QoS functions, you must re-create the configuration step-by-step.

Device

The device name of the file system.

refresh

Refreshes the active QoS configuration in the IBM Spectrum Scale daemon from the master QoS configuration that is stored in the CCR.

Device

The device name of the file system.

list [-Y]

Lists the file systems in the current cluster that are configured for QoS. Notice that this command does not require a device name as the first option.

-Y

Displays the command output in a parseable format with a colon (:) as a field delimiter. Each column is described by a header.

Note: Fields that have a colon (:) are encoded to prevent confusion. For the set of characters that might be encoded, see the command documentation of mmclidecode. Use the mmclidecode command to decode the field.

status [-Y]

Displays the current status of QoS for the specified file system.

Device

The device name of the file system.

-Y

Displays the command output in a parseable format with a colon (:) as a field delimiter. Each column is described by a header.

Note: Fields that have a colon (:) are encoded to prevent confusion. For the set of characters that might be encoded, see the command documentation of mmclidecode. Use the mmclidecode command to decode the field.

The following example displays the QoS status for file system fs1:

# mmqos filesystem status fs1
                  Configuration Repository        Daemon
                  ------------------------        ------
State:            disabled                        disabled
Version:          49                              49
Throttling:       --                              inactive
Monitoring:       --                              inactive

The command displays QoS status information for the file system both as it exists in the CCR and as it exists in the file system:

State

Indicates whether QoS is initialized, enabled, disabled, or reset.

Version

Shows how many configuration changes have occurred since the first QoS configuration action was done for this file system.

Throttling

Indicates whether any throttle rules are created.

Note: In mmqos output the value "--" indicates that the measurement is not applicable in the current context. In the preceding example, it indicates that throttling and monitoring do not occur in the CCR environment.

Monitoring

Indicates whether performance statistics are being collected.

restore --restore-file RestoreFile

Restores the QoS configuration. Do not issue this command except with the guidance of IBM® Support for a disaster recovery for QoS configuration data.

Note: This function requires that the mmsdrbackup user exit for the CCR has been configured to ensure that there is a backup that includes the QoS data as part of the CCR backup.

report

Commands for working with reports.

list

Writes a report to STDOUT.

Device

The device name of the file system.

--pool {all | PoolName}

The storage pools for which statistics are to be listed.

all

Lists statistics for all the pools that are assigned to QoS filesets.

PoolName

Lists statistics for the specified storage pool.

--seconds Seconds

Lists the I/O performance values that were collected during the previous specified number of seconds. The valid range is 1 - 999 seconds. The default is 60 seconds. The values are listed over a number of subperiods within the period that you specify. You cannot configure the number or length of subperiods. Examples of subperiods are every 5 seconds over the last 60 seconds or every 60 seconds over the last 600 seconds.

--sum-classes {yes | no}

yes

Lists the sum of the I/O performance values of all the QoS classes.

no

Lists the I/O performance of each QoS class separately. This value is the default.

--sum-nodes {yes |no}

yes

Lists the sum of the I/O performance values of all the nodes. This value is the default.

no

Lists the I/O performance of each node separately.

-Y

Displays the command output in a parseable format with a colon (:) as a field delimiter. Each column is described by a header.

Note: Fields that have a colon (:) are encoded to prevent confusion. For the set of characters that might be encoded, see the command documentation of mmclidecode. Use the mmclidecode command to decode the field.

--fine-stats-display-range FineStatsDisplayRange

Lists the fine-grained statistics that are currently available in memory. The FineStatsDisplayRange values specify the indexes of the first and last blocks of statistics in the range of blocks that you want to list, such as 0-4. If you specify only one integer, such as 2, the command lists all the blocks in memory beginning with that index and going to the end. The last line of the output lists the index of the next block in memory. You can avoid relisting fine-grained statistics by beginning the next report at this block index.

Fine-grained statistics are taken at one-second intervals and contain more information than regular statistics. The default interval is one second. You can set the interval with the stat-slot-time option. They are intended to be used as input for programs that analyze and display data, such as the example plotting program at /usr/lpp/mmfs/samples/charts/qosplotfine.pl. For the content of the statistics, see the subtopic later in this topic.

For more information, see the subtopics Analyzing regular output from the mmqos report list command and Analyzing fine-grained output from the mmqos report list command later in this topic.

Exit status

0

Successful completion.

Nonzero

A failure occurred.

Security

You must have root authority to run the mmqos command.

The node on which you enter the command must be able to run remote shell commands on any other administration node in the cluster. It must be able to do so without the use of a password and without producing any extraneous messages. For more information, see Requirements for administering a GPFS file system.

Analyzing regular output from the mmqos report list command

# mmqos report list gpfs1 --seconds 30 
QOS config::	 enabled -- --ccr:ccr-file-version=3
QOS values::	 pool=system,other=100Iops
QOS status::	 throttling active, monitoring active
QOS IO stats for pool: system
03:22:25 misc        iops=2.80000         MBs=0.06172      maxIOPS=unlimited   maxMBS=unlimited ioql=0.00136    qsdl=0.00000    et=5 wideOpen=yes(1:1:1)
03:22:25 other       iops=50.00000        MBs=197.60469    maxIOPS=50.0        maxMBS=unlimited ioql=3.04175    qsdl=22.94107   et=5 wideOpen=no(0:0:0)
03:22:30 other       iops=49.40000        MBs=197.60000    maxIOPS=50.0        maxMBS=unlimited ioql=2.99324    qsdl=27.92655   et=5 wideOpen=no(0:0:0)
03:22:35 misc        iops=2.20000         MBs=0.04219      maxIOPS=unlimited   maxMBS=unlimited ioql=0.00058    qsdl=0.00000    et=5 wideOpen=yes(1:1:1)
03:22:35 other       iops=43.20000        MBs=170.40469    maxIOPS=50.0        maxMBS=unlimited ioql=1.67852    qsdl=17.50021   et=5 wideOpen=no(0:0:0)
03:22:40 misc        iops=0.80000         MBs=0.00859      maxIOPS=unlimited   maxMBS=unlimited ioql=0.00031    qsdl=0.00000    et=5 wideOpen=yes(1:1:1)
03:23:10 misc        iops=1.20000         MBs=0.81484      maxIOPS=unlimited   maxMBS=unlimited ioql=0.00905    qsdl=0.00000    et=5 wideOpen=yes(1:1:1)
03:23:20 misc        iops=0.20000         MBs=0.00078      maxIOPS=unlimited   maxMBS=unlimited ioql=0.00006    qsdl=0.00000    et=5 wideOpen=yes(1:1:1)
mmqos: Command completed.

Only one pool is part of a throttle configuration, the system storage pool. The output indicates that IOPS and MB/s occurred only for processes in the other class and in the misc class. The meaning of the categories in each line is as follows:

First column

The time when the measurement period ends.

Second column

The QoS class for which the measurement is made.

iops=

The performance of the class in I/O operations per second.

MBs=

The performance of the class in megabytes per second.

maxIOPS=

The maxIOPS throttling limitation for the interval.

maxMBS=

The maxMBS throttling limitation for the interval.

ioql=

The average number of I/O requests in the class that are pending for reasons other than being queued by QoS. This number includes, for example, I/O requests that are waiting for network or storage device servicing.

qsdl=

The average number of I/O requests in the class that are queued by QoS. When the QoS system receives an I/O request from the file system, QoS first finds the class to which the I/O request belongs. It then finds whether the class has any I/O operations available for consumption. If not, then QoS queues the request until more I/O operations become available for the class. The Qsdl value is the average number of I/O requests that are held in this queue.

et=

The interval in seconds during which the measurement was made.

wideOpen=

Whether the throttling of the class in that interval is unlimited (1) or not (0).

You can calculate the average service time for an I/O operation as ((Ioql + Qsdl) / Iops). For a system that is running IO-intensive applications, you can interpret the value (Ioql + Qsdl) as the number of threads in the I/O-intensive applications. This interpretation assumes that each thread spends most of its time in waiting for an I/O operation to complete.

Analyzing fine-grained output from the mmqos report list command

Time, Class, Node, Iops, TotSctrs, Pool, Pid, RW, SctrI, AvgTm, SsTm, MinTm, MaxTm, AvgQd, SsQd
1480606227,misc,172.20.0.21,285,2360,system,0,R,16,0.000260,0.000197,0.000006,0.005894,0.000001,0.000000
1480606228,misc,172.20.0.21,631,5048,system,0,R,16,0.000038,0.000034,0.000006,0.002463,0.000000,0.000000
1480606239,other,172.20.0.21,13,112,system,26724,R,16,0.000012,0.000000,0.000008,0.000022,0.000001,0.000000
1480606239,other,172.20.0.21,1,512,system,26724,R,512,0.000375,0.000000,0.000375,0.000375,0.000002,0.000000
1480606239,other,172.20.0.21,30,15360,system,26724,W,512,0.000910,0.000004,0.000451,0.002042,0.000001,0.000000
1480606240,misc,172.20.0.21,5,48,system,14680072,W,16,0.000135,0.000000,0.000025,0.000258,0.000000,0.000000
1480606240,other,172.20.0.21,4,32,system,26724,R,16,0.000017,0.000000,0.000009,0.000022,0.000001,0.000000
1480606240,other,172.20.0.21,48,24576,system,26724,W,512,0.000916,0.000006,0.000490,0.002141,0.000001,0.000000
1480606241,other,172.20.0.21,10,80,system,26724,R,16,0.000017,0.000000,0.000007,0.000039,0.000001,0.000000
1480606241,other,172.20.0.21,34,17408,system,26724,W,512,0.000873,0.000007,0.000312,0.001957,0.000001,0.000000
1480606241,misc,172.20.0.21,12,104,system,15597572,W,16,0.000042,0.000000,0.000024,0.000096,0.000000,0.000000
1480606241,misc,172.20.0.21,1,512,system,15597572,W,512,0.000192,0.000000,0.000192,0.000192,0.000000,0.000000
1480606241,misc,172.20.0.21,9,72,system,14680071,W,16,0.000029,0.000000,0.000024,0.000040,0.000000,0.000000
## conti=4

## conti=4