Planning for Quality of Service for I/O operations (QoS)
With the Quality of Service for I/O operations (QoS) capability, you can run a GPFS maintenance command without the risk of it dominating file system I/O performance and significantly delaying other tasks.
The GPFS maintenance commands are a group of about 20 potentially long-running, I/O-intensive commands that can generate hundreds or thousands of requests for I/O operations per second. This high demand for I/O can greatly slow down normal tasks that are competing for the same I/O resources. With QoS, you can assign an instance of a maintenance command to a QoS class that has a lesser I/O priority. Although the instance of the maintenance command now takes longer to run to completion, normal tasks have greater access to I/O resources and run more quickly. The maintenance commands are listed in the topic mmchqos command.
- QoS requires the file system to be at V4.2.0.0 or later.
- QoS works with asynchronous I/O, memory-mapped I/O, cached I/O, and buffered I/O. However, with direct I/O, QoS counts the I/O operations per second (IOPS) but does not regulate them.
- When you change allocations, mount the file system, or re-enable QoS, a brief delay due to reconfiguration occurs before QoS starts applying allocations.
To set up QoS, you allocate shares of I/O operations per second (IOPS) to two QoS classes, maintenance and other. Each storage pool has these two classes. The maintenance class is the class in which the GPFS maintenance commands run by default. Typically you assign a smaller share of IOPS to this class so that the commands in it do not overwhelm the file system with I/O requests. The other class is the class to which all other processes belong. Typically you assign an unlimited number of IOPS to this class so that normal processes have greater access to I/O resources and finish more quickly.
When a process issues an I/O request, QoS first finds the QoS class to which the process belongs. It then finds whether the class has any I/O operations available for consumption. If so, QoS allows the I/O operation. If not, then QoS queues the request until more I/O operations become available for the class. For a particular storage pool, all instances of maintenance commands that are running concurrently compete for the IOPS that you allocated to the maintenance class of that storage pool. Similarly, all processes that belong to the other class compete for the IOPS that you allocated to the other class of that storage pool. If no processes in the maintenance class are running, processes in the other class can also compete for the maintenance IOPS.
If you do not configure a storage pool for QoS, then processes compete for I/O operations in that storage pool in the normal way without interference.
By default, for both the maintenance class and the other class, QoS divides the IOPS allocation equally among the nodes of the cluster that have the file system mounted. For example, you might assign 1000 IOPS to the maintenance class of storage pool SP1. If your allocation applies to all the nodes in the cluster (the default setting) and the cluster has five nodes, then the maintenance class of each node gets an allocation of 200 IOPS for storage pool SP1. At run time, on each node all processes that belong to the maintenance class and that access storage pool SP1 compete for the 200 IOPS.
You can also divide IOPS among a list of nodes, a user-defined node class, or the nodes of a remote cluster that is served by the file system.
To get started with QoS, follow the steps that are outlined in the topic Setting the Quality of Service for I/O operations.
Use the mmchqos command to allocate IOPS to the QoS classes maintenance and other. You can set up allocations for one pool or several pools in one call to the command. You can designate a single node, several nodes, or all the nodes in the current cluster to participate in the QoS operations. You can also designate the nodes in a remote cluster to participate. To preserve a particular configuration, you can set up a stanza file that the mmchqos command reads when it runs.
You can reset or change IOPS allocations at any time without unmounting the file system. You can also disable QoS at any time without losing your IOPS allocations. When you reenable QoS, it resumes applying the allocations.
- You can allocate shares of IOPS separately for each storage pool.
- QoS divides each IOPS allocation equally among the specified nodes that mounted the file system.
- Allocations persist across unmounting and remounting the file system.
- QoS stops applying allocations when you unmount the file system and resumes when you remount it.
Use the mmlsqos command to display the consumption of IOPS. The command displays data for subperiods within the period that you specify. You can configure the type of information that is displayed. You can display information for specified storage pools or for all storage pools. You can display information for various time periods, from 1 second up to about 15 minutes. You can display I/O performance for each QoS class separately or summed together. You can also display the I/O performance of each node separately or summed across the participating nodes.