Configuring Workload Manager to consolidate workloads

Workload Manager (WLM) gives you control over the resources used by jobs on your system.

A default WLM configuration template exists on every installed AIX® operating system. The following procedure updates the WLM configuration template to implement a resource-management policy on a shared server. The resulting configuration can be used as a starting point for testing. Exactly how you configure WLM will depend on the workload and policy requirements for your environment.

Note:

Efficient use of WLM requires extensive knowledge of existing system processes and performance. Repeated testing and tuning will probably be needed before you can develop a configuration that works well for your workload. If you configure WLM with extreme or inaccurate values, you can significantly degrade system performance.
The process of configuring WLM is simpler when you already know one or more of the classification attributes of a process (for example, user, group, or application name). If you are unfamiliar with the current use of resources, use a tool such as topas to identify the processes that are the primary resource users and use the resulting information as the starting point for defining classes and rules.
The following scenario assumes you are familiar with the basic Workload Manager concepts as described in Workload management concepts.

The WLM configuration files exist in the /etc/wlm/ConfigurationName directory. Each subclass for each superclass is defined in a configuration file named /etc/wlm/ConfigurationName/SuperClassName. For more information about these files, see the Files Reference.

In the following procedure, you consolidate the workloads from two separate department servers onto one larger server. This example edits the configuration files, but you can also create a configuration using SMIT (use the smit wlmconfig_create fast path). Briefly, in this procedure, you will do the following:

Identify the resource requirements of the applications you want to consolidate. This will help you determine how many applications you can move to the larger server.
Define tiers, as well as resource shares and limits, to begin testing with the consolidated workload.
Fine-tune the configuration until you achieve your desired results.

The information in this how-to scenario was tested using specific versions of AIX. The results you obtain might vary significantly depending on your version and level of AIX.

Step 1. Identify application requirements

In this scenario, the workload is typical of what you might see on a database server. Assume the jobs fall into the following general categories:

Listeners: These are processes that sleep most of the time and wake up periodically in response to a request. Although these processes do not consume a lot of resources, response time can be critical.
Workers: These are processes that do the work on behalf of a request, whether the request is local or remote. These processes probably use a lot of CPU time and memory.
Reporters: These are processes that do automated tasks. They might require a lot of CPU time or memory, but they can tolerate a slower response time.
Monitors: These are processes that typically run periodically to verify the state of the system or applications. These processes might use a significant amount of resource, but only for a short time.
Commands: These are commands or other applications that system users might run at any time. Their resource requirements are unpredictable.

In addition to this work, scheduled jobs fall into one of the following categories:

SysTools: These are processes that perform automated tasks. These jobs are not critical to system operation but need to run periodically and within certain time constraints.
SysBatch: These are processes that run infrequently, are not critical to system operation, and need not finish in a timely manner.

The first step of creating a configuration is to define classes and rules. In the following steps, you will use the general job categories listed above to define your classes. Use the following procedure:

Make a new configuration within the /etc/wlm directory called MyConfig using the following command:
```
mkdir /etc/wlm/MyConfig
```
Copy the template files into the /etc/wlm/MyConfig directory using the following command:
```
cp -pr /etc/wlm/template/* /etc/wlm/MyConfig
```
To create the super classes, use your favorite editor to modify the /etc/wlm/MyConfig/classes file to contain the following:
```
System:

Default:

DeptA:

DeptB:

SysTools:

SysBatch:
```
As a starting point, you define one superclass for each department (because two departments will be sharing the server). The SysTool and SysBatch super classes will handle the scheduled jobs outlined in the general categories above. The System and Default super classes are always defined.
Within the MyConfig directory, create a directory for each the DeptA and DeptB super classes. (When creating a configuration, you must create a directory for every superclass that has subclasses.) In the following step, you define five subclasses (one for each category of work) for each department's superclass.
To create subclasses for each general category of jobs, edit the /etc/wlm/MyConfig/DeptA/classes and /etc/wlm/MyConfig/DeptB/classes files to contain the following:
```
Listen:
        
Work:
        
Monitor:
        
Report:
        
Command:
        
```
Note: The contents of the classes file can be different for each superclass.

After the classes are identified, in the following step, you create the classification rules that are used to classify processes at the superclass and subclass levels. For the sake of simplicity, assume that all applications run from known locations, that all processes from one department run under the deptA UNIX group, and that processes from the other department run under the deptB UNIX group.
To create the superclass assignment rules, modify the /etc/wlm/MyConfig/rules file to contain the following:
```
DeptA - - deptA - -
DeptB - - deptB - -
SysTools - root,bin - /usr/sbin/tools/* -
SysBatch - root,bin - /usr/sbin/batch/* -
System - root - - -
Default - - - - -
```
Note: If more than one instance of the same application can be running and all classification attributes (other than the tag) are the same, use the wlmassign command or wlm_set_tag subroutine to differentiate between them by assigning them to different classes.

To create more specific subclass rules, create the /etc/wlm/MyConfig/DeptA/rules and /etc/wlm/MyConfig/DeptB/rules files with the following content:

Listen - - - /opt/myapp/bin/listen* -
Work - - - /opt/myapp/bin/work* -
Monitor - - - /opt/bin/myapp/bin/monitor -
Report - - - /opt/bin/myapp/report* -
Command - - - /opt/commands/* -

To determine the resource-consumption behavior of each class, start WLM in passive mode using the following command:
```
wlmcntrl -p -d MyConfig
```
After starting WLM in passive mode, you can run each application separately at first to gain a finer perspective of its resource requirements. You can then run all applications simultaneously to better determine the interaction among all classes.

An alternative method of identifying the application resource requirements might be to first run WLM in passive mode on the separate servers from which you are consolidating applications. The disadvantages to this approach are that you would have to re-create the configurations on the larger system, and the required percentage of resources will likely be different on the larger system.

Step 2. Define Tiers, Shares, and Limits

A WLM configuration is an implementation of a resource-management policy. Running WLM in passive mode provides information that helps you determine whether your resource-management policy is reasonable for the given workload. You can now define tiers, shares, and limits to regulate your workload based on your resource-management policy.

For this scenario, assume you have the following requirements:

The System class must have the highest priority and be guaranteed access to a percentage of system resources at all times.
The SysTools class must have access to a certain percentage of resources at all times, but not so much that it will significantly impact the applications that are running in DeptA and DeptB.
The SysBatch class cannot interfere with any of the other work on the system.
DeptA will receive 60% of the available resources (meaning resources that are available to the classes with shares) and DeptB will receive 40%. Within DeptA and DeptB:
- Processes in the Listen class must respond to requests with a low latency, but must not consume a lot of resources.
- The Work class must be allowed to consume the most resources. The Monitor and Command classes must consume some resource, but less than the Work class.
- The Report class cannot interfere with any of the other work.

In the following procedure, you define tiers, shares, and limits:

To create the superclass tiers, use your favorite editor to modify the /etc/wlm/MyConfig/classes file to contain the following:
```
System:

Default:

DeptA:
        localshm = yes
        adminuser = adminA
        authuser = adminA
        inheritance = yes

DeptB:
        localshm = yes
        adminuser = adminB
        authuser = adminB
        inheritance = yes

SysTools:
        localshm = yes

SysBatch:
        tier = 1
        localshm = yes 
```
The SysBatch superclass is put in tier 1 because this class contains very low-priority jobs that you do not want to interfere with the rest of the work on the system. (When a tier is not specified, the class defaults to tier 0.) Administration of each department's superclass is defined by the adminuser and authuser attributes. The inheritance attribute is enabled for DeptA and DeptB. All new processes started in a class with inheritance will remain classified in that class.
To create subclass tiers for each group of jobs, modify the /etc/wlm/MyConfig/DeptA/classes and /etc/wlm/MyConfig/DeptB/classes files to contain the following:
```
Listen:
        
Work:
        
Monitor:
        
Report:
        tier = 1
Command:
        
```
To assign the initial shares for the superclasses, edit the /etc/wlm/MyConfig/shares file to contain the following:
```
DeptA:
        CPU = 3
        memory = 3

DeptB:
        CPU = 2
        memory = 2 
```
Because you assigned a CPU total of 5 shares, DeptA processes will have access to three out of five shares (or 60%) of the total CPU resources and DeptB processes will have access to two out of five (or 40%). Because you did not assign shares to the SysTools, System, and Default classes, their consumption targets will remain independent from the number of active shares, which gives them higher-priority access to resources than the DeptA and DeptB (until their limit is reached). You did not assign the SysBatch class any shares because it is the only superclass in tier 1, and therefore any share assignment is irrelevant. Jobs in the SysBatch class can only consume resources that are unused by all classes in tier 0.
To assign the initial shares for the subclasses, edit the /etc/wlm/MyConfig/DeptA/shares and /etc/wlm/MyConfig/DeptB/shares files to contain the following:
```
Work:
        CPU = 5
        memory = 5

Monitor:
        CPU = 4
        memory = 1

Command:
        CPU = 1
        memory = 1 
```
Because you did not assign shares to the Listen class, it will have the highest-priority access (in the superclass) to resources when it requires them. You assigned the largest number of shares to the Work class because it has the greatest resource requirements. Accordingly, you assigned shares to the Monitor and Command classes based on their observed behavior and relative importance. You did not assign shares to the Report class because it is the only subclass in tier 1, and therefore any share assignment is irrelevant. Jobs in the Report class can only consume resources that are unused by subclasses in tier 0.
In the following step of this example, you assign limits to classes that were not assigned shares. (You can also assign limits to classes with shares. See Managing Resources with WLM for more information.)
To assign limits to the superclasses, edit the /etc/wlm/MyConfig/limits file to contain the following:
```
Default:
        CPU = 0%-10%;100%
        memory = 0%-10%;100%

SysTools:
        CPU = 0%-10%;100%
        memory = 0%-5%;100%

System:
        CPU = 5%-50%;100%
        memory = 5%-50%;100% 
```
You assigned soft maximum limits to the System, SysTools, and Default classes to prevent them from significantly interfering with other work on the system. You assigned minimum limits to the System class for CPU and memory because this class contains processes that are essential to system operation, and it must be able to consume a guaranteed amount of resource.
To assign limits to the subclasses, edit the /etc/wlm/MyConfig/DeptA/limits and /etc/wlm/MyConfig/DeptB/limits files to contain the following:
```
Listen:
        CPU = 10%-30%;100%
        memory = 10%-20%;100%

Monitor:
        CPU = 0%-30%;100%
        memory = 0%-30%;100% 
```
Note: The limits can be different for each subclass file.

You assigned soft maximum limits to the Listen and Monitor classes to prevent them from significantly interfering with the other subclasses in the same superclass. In particular, you do not want the system to continue accepting requests for jobs within the Work class, if the Work class does not have access to the resources to process them. You also assigned minimum limits to the Listen class to ensure fast response time. The minimum limit for memory ensures that pages used by this class will not be stolen by page replacement, resulting in faster execution time. The minimum limit for CPU ensures that when these processes can be run, they will have the highest-priority access (in the superclass) to the CPU resources.

Step 3. Fine-tune the Workload Manager configuration

Monitor the system using the wlmstat command and verify that the regulation done by WLM aligns with your goals and does not unduly deprive some applications of resources while others get more than they should. If this is the case, adjust the shares and refresh WLM.
As you monitor and adjust the shares, limits, and tier numbers, decide whether you want to delegate the administration for the subclasses for some or all of the superclasses. The administrator can then monitor and set up the subclass shares, limits, and tier number.

The administrator of each superclass can repeat this process for the subclasses of each superclass. The only difference is that WLM cannot run in passive mode at the subclass level only. The subclass configuration and tuning has to be done with WLM in active mode. One way not to impact users and applications in the superclass is to start the tier number, and the shares and limits for the subclasses at their default value ('-' (hyphen) for shares, 0% for minimum, and 100% for soft and hard maximum). With these settings, WLM will not regulate the resource allocation between the subclasses.

For more information

Workload Manager.
Workload Management.
Workload Management Diagnosis in Performance management.
The descriptions for the classes, limits, rules, and shares files in the Files Reference.
The topas, wlmassign, wlmcheck, wlmcntrl, and wlmstat.
The WLM subroutine descriptions, especially wlm_set_tag.