Configure Tivoli Dynamic Workload Broker and EWLM to produce efficient job dispatching and scheduling

Tivoli Dynamic Workload Broker is a dispatching product that can use dynamic resource information as well as recommendations from other products to determine the best systems to which new jobs should be dispatched. Enterprise Workload Manager is an autonomic workload manager that uses its many resource allocation and recommendation abilities to help achieve user-defined goals. This article describes the related parts of these products and explains in detail how they have been designed to work together.

Before you start

While we will briefly describe some of the necessary components of the interaction between the products, we won't attempt to explain the full capability of either product; for that, please consult the appropriate documentation:

• For Tivoli Dynamic Workload Broker:
  • IBM Tivoli Dynamic Workload Broker Installation and Configuration, V1.1 (SC32-2282-00, October 2006)
  • IBM Tivoli Dynamic Workload Broker User Guide V1.1 (SC32-2281-00, October 2006)
  
  
• For Enterprise Workload Manager:
  • IBM Enterprise Workload Manager (IBM Redbook SG24-6350, March 2005)
  • The EWLM Infocenter at http://publib.boulder.ibm.com/infocenter/eserver/v1r2/index.jsp

You should have a good understanding of both products before reading this article because all configuration steps in this article assume that you can configure a working Tivoli Dynamic Workload Broker environment as well as a working EWLM environment. If this is not the case, you should consult the previously mentioned resources first, before attempting to use the configurations we describe.

All information refers to Tivoli Dynamic Workload Broker version 1.1 (released in October 2006) and EWLM version V1R2 (released November 2005) unless otherwise specified.

We'll start by introducing you to Tivoli Dynamic Workload Broker.

Introducing Tivoli Dynamic Workload Broker

The IBM® Tivoli Dynamic Workload Broker is innovative workload automation software that helps you convert your existing technology infrastructures into dynamic, virtualized environments.

Tivoli Dynamic Workload Broker improves business efficiency and reduces Total Cost of Ownership (TCO) by automatically adapting job execution to environment changes, distributing workloads to "best available" resources across a dynamically shifting, cross-enterprise resource pool, and drastically reducing the labor-intensive process of manually planning job assignment to each resource.

Tivoli Dynamic Workload Broker introduces an abstraction layer between the workload and the physical resources allowing their dynamic allocation. It optimizes capacity of the IT infrastructure to execute more workload with less hardware. Based on current resource load and usage needs, jobs are submitted to the less loaded resource.

Tivoli Dynamic Workload Broker key features are:

• Automatic discovery of IT environment resource
• Intelligent job distribution based on job requirements and administration policies
• Computing resource utilization optimization
• Use load balancing information provided by EWLM to make better job routing decisions to optimize resource utilization or to achieve the desired business service level objectives (SLA) (we focus on this in the article)

A typical Tivoli Dynamic Workload Broker scheduling environment consists of a set of Tivoli Dynamic Workload Broker agents and a Tivoli Dynamic Workload Broker server, as shown in Figure 1.

The Tivoli Dynamic Workload Broker Server manages job submission requests converting job resource requirements into appropriate resource allocation requests and distributing workloads to "best available" resources.

The Tivoli Dynamic Workload Broker Agent is the component responsible for:

• Discovering resources available on each computer system and forwarding this information to the Tivoli Dynamic Workload Broker server.
• Managing the job submission requests received from the Tivoli Dynamic Workload Broker server by running and monitoring jobs on the local system. The information on the job status is returned to the Tivoli Dynamic Workload Broker server. It is the Tivoli Dynamic Workload Broker agent that is ARM-enabled.

Introducing EWLM

The Enterprise Workload Manager is a robust performance-management tool that lets you monitor and manage work that runs within enterprise environments. EWLM has a variety of management abilities in its repertoire including:

• Advising load balancers (this management action is the focus of this article)
• Moving CPU resources across virtual servers
• Provisioning new servers to needy server tiers

EWLM management domains consist of a collection of EWLM managed servers and a domain manager. The domain manager is the central point of control for a domain because it coordinates the activation of policies on the managed servers and the collection of performance data. A managed server is a server whose work requests are monitored by EWLM. The managed server sends performance data to the domain manager. The domain manager includes the EWLM user interface referred to as the EWLM Control Center.

The EWLM Managed Server gathers system statistics from the underlying operating system. If the applications running on the system are ARM instrumented (Application Response Measurement) the managed server also gathers application statistics. The Tivoli Dynamic Workload Broker agent is ARM instrumented to provide application information for EWLM to use when calculating weights for efficient job dispatching.

EWLM and Tivoli Dynamic Workload Broker interact in three different ways:

• Monitoring: The Tivoli Dynamic Workload Broker agent is ARM-instrumented to allow EWLM to classify and closely monitor the work running on the agent systems.
• Advanced load balancing/job dispatching: Tivoli Dynamic Workload Broker communicates with EWLM through the Server/Application State Protocol (SASP) to get weights that indicate the best distribution of Tivoli Dynamic Workload Broker Agents to which to route incoming jobs.
• EWLM autonomic management: If on a supported autonomic management platform, EWLM uses other resource allocation methods to help achieve the administratively-configured goals of the Tivoli Dynamic Workload Broker jobs (as you'll see in the section on resource allocation).

For more general information about EWLM, please visit the EWLM InfoCenter.

Planning the interaction

When planning for the interaction of Tivoli Dynamic Workload Broker and EWLM, keep in mind the supported platforms of each product and the communication between them to ensure the environment is supported by both products.

Platform support

The Tivoli Dynamic Workload Broker Server is not a job-processing entity; therefore, it is not ARM instrumented and does not need to be on a EWLM-monitored machine. The Agent Manager is also not a job-processing entity and need not be on a EWLM-monitored machine.

The Tivoli Dynamic Workload Broker agent is ARM instrumented and processes the Tivoli Dynamic Workload Broker jobs; therefore, it should be on a machine monitored with an EWLM Managed Server. Because the Tivoli Dynamic Workload Broker agent will also be a part of the EWLM load balancing algorithm, it should be on a machine with an operating system with EWLM Managed Server load balancing support.

Given these requirements, it is important that the Tivoli Dynamic Workload Broker Agent and EWLM Managed Server be on the systems supported by the appropriate components of both products. Table 1 is a matrix of this cross-product support (EWLM Managed Server load balancing support and Tivoli Dynamic Workload Broker agent support).

The table was created from the supported operating system list of Tivoli Dynamic Workload Broker version 1.1 and EWLM Release V1R2. Please consult appropriate documentation to determine supported operating systems for future releases of either product.

Communication between products

As discussed further in the next section, the Tivoli Dynamic Workload Broker server connects to the EWLM Domain Manager to receive SASP load balancing recommendations. This is a standard TCP connection (typically on port 3860) for binary communication. If the Tivoli Dynamic Workload Broker server is in a less secure zone than the Domain Manager, special provisions would need to be made to ensure the Tivoli Dynamic Workload Broker connection can go through the firewall to reach the Domain Manager.

EWLM load balancing recommendations in Tivoli Dynamic Workload Broker

While the section on introducing EWLM enumerated three points of interaction between EWLM and Tivoli Dynamic Workload Broker, in this article we're concentrating on Tivoli Dynamic Workload Broker's use of EWLM load balancing weights for efficient job dispatching. Figure 3 demonstrates an example environment involving this interaction.

In Figure 3, the flow of Tivoli Dynamic Workload Broker jobs is illustrated by the bold, orange arrows. EWLM's communication between its managed servers and its domain manager is shown in blue. The long-lived SASP connection between the Tivoli Dynamic Workload Broker Server and the EWLM Domain Manager is shown in green.

During the connection illustrated by the green line, the Tivoli Dynamic Workload Broker server registers the Tivoli Dynamic Workload Broker agents as group members with the EWLM Domain Manager. The Tivoli Dynamic Workload Broker Server then receives weight updates from the Domain Manager every 30 seconds. These weight updates are immediately applied and affect the manner in which Tivoli Dynamic Workload Broker dispatches jobs until the next weight update is received.

The following steps must be followed to start the interaction:

1. Enable EWLM load balancing.
2. Install the EWLM plug-in at Tivoli Dynamic Workload Broker installation.
3. Set EWLM as the optimization policy in the Tivoli Dynamic Workload Broker job definition.

The remaining steps are highly recommended for the most efficient job dispatching interaction:

4. Enable ARM on the Tivoli Dynamic Workload Broker Agent.
5. Define EWLM classification and service goal criteria.

Starting the interaction

Now let's look at starting the interaction.

Turning on EWLM load balancing

EWLM's process of generating load balancing weights that may be used for the efficient dispatching of jobs must be turned on and configured prior to use. This can be done using the Configuration Wizard at EWLM installation or through the use of the changeDM command after EWLM has already been installed. The typical configuration uses the following parameters:

• -lbp (load balancing port): 3860
• -ma (management address): 0.0.0.0

A value of 0.0.0.0 is used for the management address to permit the domain manager to accept connections to the load balancing port on any valid IP address (this is important when using machines with more than one network interface).

Enabling Tivoli Dynamic Workload Broker to receive EWLM load balancing weights

Next you have to enable Tivoli Dynamic Workload Broker to accept the load balancing weights from EWLM during the Tivoli Dynamic Workload Broker installation process.

It's "install time" steps

EWLM enablement is a Tivoli Dynamic Workload Broker extension that must be installed with the Tivoli Dynamic Workload Broker Server. When installed, the EWLM plug-in for Tivoli Dynamic Workload Broker will be integrated into the Server, registering the Tivoli Dynamic Workload Broker agents as load balancing group members in EWLM and receiving new weights every 30 seconds.

As shown in Figure 4, the administrator simply selects "EWLM enablement" from the feature list when installing Tivoli Dynamic Workload Broker.

After EWLM enablement is selected, the EWLM enablement configuration panel appears allowing the administrator to tell Tivoli Dynamic Workload Broker how to connect to the EWLM Domain Manager (Figure 5).

For this to work, the following fields must be completed:

• EWLM Domain Manager Name: This is the unique identifier that the Tivoli Dynamic Workload Broker server uses to identify itself with the EWLM Domain Manager. Care should be taken to ensure this name is not used by any other products connecting to EWLM for load balancing weights.
• EWLM Domain Manager Address: This is the IP address or resolvable host name of the EWLM Domain Manager. The value used here should be the same value used for the -ma parameter of the EWLM installation (unless 0.0.0.0 was used for the -ma parameter).
• EWLM Domain Manager Port (LBP): This is the SASP port for the EWLM Domain Manager. The value used here should be the same value used in the -lbp parameter during EWLM configuration.
• EWLM weight scope: This value should always be set to Application.
• EWLM refresh interval: Interval that determines how often the Tivoli Dynamic Workload Broker Server tries to update Domain Manager group settings (if needed). Default value is 90 seconds.

Job definitions

To make Tivoli Dynamic Workload Broker dispatch jobs using the weights provided by the EWLM, it is necessary to define EWLM as the optimization policy in the job definition as shown in Listing 1:

<jsdl:optimization name="JPT_EWLM">
     <jsdl:ewlm/>
</jsdl:optimization>

Enabling ARM on the Tivoli Dynamic Workload Broker Agent

To enable ARM on the Tivoli Dynamic Workload Broker Agent requires the following two steps:

1. Set the arm.enabled variable on the Job Execution Agent.

Set arm.enabled=true in the JobExecutionAgentConfig.properties file. The JobExecutionAgentConfig.properties file can be found at the following path: <ITDWB_Agent_InstallDir>\ep\runtime\agent\subagents\JobExecutionAgent\.

2. Set the ARM native library path correctly.

The Tivoli Dynamic Workload Broker agent exploits IBM's Java™ ARM implementation, which needs a native ARM library. Table 2 describes the native library name and location for the collectively supported environments shown in the section on platform support.

In order for Tivoli Dynamic Workload Broker to find this library, the native library path must be edited to include the directory of the appropriate library listed above.

• For AIX, the LD_LIBRARY_PATH environment is used to define the native library path; for example, LD_LIBRARY_PATH=/usr/lib.
• For Windows, the directory containing the JNI library should be defined by the PATH environment variable; for example, PATH=%PATH%;<ewlm_root>\ms.
• For SLES 9, the directory containing the JNI library should be defined by the PATH environment variable; for example, PATH=/usr/lib.

If the IBM JNI native library could not be found, the Tivoli Dynamic Workload Broker Agent will not be ARM instrumented. More information can be found in the appropriate ARM-related documentation from the EWLM InfoCenter.

EWLM classification of Tivoli Dynamic Workload Broker jobs

EWLM reports unclassified Tivoli Dynamic Workload Broker work without any configuration from the administrator. However, EWLM classifies work in the enterprise environment to understand management importance, identify work for goal setting, and to provide more detailed reporting. To classify work being done in different applications, EWLM must first understand the information each instrumented application exposes for classification. This is provided in an Application filter xml file (named TDWB_Agent.xml) that is shipped with the Tivoli Dynamic Workload Broker Server. This file must be imported into the EWLM Control Center by using the Import button on the Applications panel (the Applications panel is reached by using the Applications link under Set up).

Import the file named TDWB_Agent.xml. TDWB_Agent.xml can be found at the following path in the Tivoli Dynamic Workload Broker Server directory space: <server_installation_directory>/EWLM/samples.

After the Tivoli Dynamic Workload Broker application file has been imported, you will then see it in the list of applications. This step also allows the Tivoli Dynamic Workload Broker Agent to be used for classifying work in the EWLM policy.

Creating the EWLM policy

There is extensive documentation on the general creation of EWLM policies; therefore, we only cover the parts pertaining to the Tivoli Dynamic Workload Broker interaction. Before defining Tivoli Dynamic Workload Broker classification criteria in the EWLM policy for work in the current environment, the application should also be added to the policy as one of the applications that can classify work for this policy. This is done by using the Add button in the Applications panel of the policy creation screens (the panel can be reached by selecting the Domain Policies link under Set up and then creating a new or editing an existing Domain Policy).

Select the Tivoli Dynamic Workload Broker Agent application to add from the drop-down menu, and then press OK:

Now that the Tivoli Dynamic Workload Broker Agent application has been added to the policy, you are able to select it as one of the applications used to classify work when creating EWLM service classes and transaction classes.

Creating EWLM service classes

Much of the remainder of this article focuses on how to classify the Tivoli Dynamic Workload Broker work into EWLM transaction classes; however, each transaction class must be in an EWLM service class in order to be actively managed by EWLM. The service classes must exist prior to the creation of the transaction classes. The service classes for Tivoli Dynamic Workload Broker work are no different than any other service classes defined in EWLM, so existing documentation should be consulted.

While EWLM permits a variety of goal types to be used in its service classes, some of the most popular goal types for transaction classes are response-time-oriented goals. These goals can be very effective for short transactions or those that must be executed in a particular time frame; however, administrators may find that velocity goals specifying the comparative fraction of resources for the service class may be more appropriate for longer running transactions.

Creating EWLM transaction classes

After the Tivoli Dynamic Workload Broker Agent application has been added and service classes have been created, transaction classes can now be created. The Transaction Class creation panel is shown in Figure 10. Note the Tivoli Dynamic Workload Broker application is selected from the drop-down menu and EWLM has already created a default transaction class. The New button should be used to create new transaction classes with Tivoli Dynamic Workload Broker classification criteria.

After selecting the New button, the New Transaction Class panel is displayed, giving the administrator the ability to specify the transaction class name, the position of classification, and the associated service class. To determine appropriate values for these fields, check the documentation mentioned at the start of this article. To begin setting the classification rules, the New button under Rules must be selected.

We'll cover creating classification rules for the Tivoli Dynamic Workload Broker/EWLM interaction in detail later.

Tivoli Dynamic Workload Broker/EWLM joint classification criteria

Classification of the Tivoli Dynamic Workload Broker work can be done in several ways. Table 3 briefly describes the current criteria that may be used for classification of Tivoli Dynamic Workload Broker work (each is explained in detail in later subsections).

* Note that the Category classification criteria is named Category1, Category2, and Category3 in EWLM, but has no number when first described in the Tivoli Dynamic Workload Broker job definition. When using categories in the Tivoli Dynamic Workload Broker job definition, the first Tivoli Dynamic Workload Broker category defined is EWLM's Category1, the second Tivoli Dynamic Workload Broker Category is EWLM's Category2, and so on. This translation between products is done automatically.

It is not necessary to make EWLM transaction classes using every way of classifying Tivoli Dynamic Workload Broker work. The three different classification methods are present to provide maximum flexibility to the administrator, for example:

• You can create a silver transaction class that is a combination of any of these three, for example: all work executable work (EWLM:Transaction name = executable) with job name of "silver" (JobName = "silver").
• You could create a silver transaction class that is simply based on one of the criteria, for example: all work with job name of "silver" (JobName = "silver").

Creating EWLM transactions classified by Tivoli Dynamic Workload Broker application name

Tivoli Dynamic Workload Broker lets its users create different types of jobs to be executed by the Tivoli Dynamic Workload Broker agents. The two types currently supported are

• executable
• j2ee

If the user would like to add new job types, he may create a pluggable bundle for Tivoli Dynamic Workload Broker and specify this job type in the Job Definition file. The type of job must be provided as a name attribute of the application element in the Job Definition file.

EWLM recognizes this Tivoli Dynamic Workload Broker application name as a transaction type. To create an EWLM rule using the Tivoli Dynamic Workload Broker application name, the administrator needs to select EWLM:Transaction name to be in the left side of the rule, the "==" for the operator, and enter the appropriate Tivoli Dynamic Workload Broker application name value for the right side of the rule equation. Figure 13 shows a rule defined using EWLM transaction name equal to "executable" (see other possible Tivoli Dynamic Workload Broker application name values in Table 3).

The new rule is displayed in the final panel for the transaction class definition:

Creating EWLM transactions classified by job name

Classifying using Tivoli Dynamic Workload Broker job name is similar to creating a transaction class using the Tivoli Dynamic Workload Broker application name attribute (as described in the previous section). First, the job name should be configured in the Tivoli Dynamic Workload Broker job definition. In the example in Figure 15, the job name "DBCleanUp" has been configured.

To create an EWLM rule using the Tivoli Dynamic Workload Broker job name, the administrator needs to select "Job Name" to be in the left side of the rule, the "==" for the operator, and enter the configured Tivoli Dynamic Workload Broker job name for the right side of the rule equation:

And here's the transaction class and newly created rule:

Creating EWLM transactions classified by categories

In addition to Tivoli Dynamic Workload Broker application name and job name, Tivoli Dynamic Workload Broker permits administrators to classify jobs using the category attribute in the job definition. It is possible to define up to three different categories for each job. Figure 18 shows the definition of a job belonging to two different categories -- it is a "Financial" job and it is a "Critical" job.

To create an EWLM rule using Tivoli Dynamic Workload Broker categories, the administrator needs to select the appropriate EWLM category for the left side of the rule equation. If multiple Tivoli Dynamic Workload Broker categories are used, rules combining the appropriate EWLM categories can be created.

In the example in Figure 19, "Category1" was first selected with a value of "Financial" on the right side of the rule equation. The logical "AND" operator is then dragged over to the end of the rule equation to create the opportunity for a second, joint rule. In the second rule, "Category2" was selected to be in the left side of the rule and the value of "Critical" is used in the right side of the rule.

The resulting transaction class is then displayed:

Confirming the interaction from the EWLM Control Center

EWLM's Control Center displays a list of all load balancers and schedulers that connect to the Domain Manager through SASP. To see the current state of the Tivoli Dynamic Workload Broker connection to EWLM, simply go to the load balancing panel of the EWLM Control Center.

Find the Tivoli Dynamic Workload Broker Server in the list identified by its IP address and the load balancing identifier. The load balancing identifier is the value used for the EWLM Domain Manager Name when installing the EWLM plug-in during the Tivoli Dynamic Workload Broker installation (in this case it was "TDWB_EWLM"). Select the Tivoli Dynamic Workload Broker Server and the Details action to get to the Load Balancer Details panel of the Tivoli Dynamic Workload Broker Server. The Load Balancer Details panel displays all of the groups and group members registered by the Tivoli Dynamic Workload Broker Server. The current state and weight of each group member is also included in the report. An example is provided in Figure 22:

Application level load balancing

EWLM's load balancing algorithm relies on ARM instrumentation to get application information to use when generating load balancing weights (called application level load balancing). If any member in the registered group has not reported ARM statistics, only system statistics are used when computing weights for the entire group (called system level load balancing). While system load balancing weights are still effective, factors such as application level failures cannot be factored into the calculation unless the group is at the application load balancing level.

An administrator can determine the load balancing level of each group in the EWLM Control Center's load balancing panel by looking at the Group Load Balancer Type field of the group. If the Group Load Balancer Type has a value of "System" the following actions can be taken to change it to "Application":

• Ensure that "Application" was chosen for the "Weight Scope" when installing Tivoli Dynamic Workload Broker (in this section) and that ARM is enabled on the Tivoli Dynamic Workload Broker Agent (in this section).
• Forcefully send a transaction to each member in the group to ensure that the corresponding EWLM Managed Servers have reported ARM statistics to the Domain Manager.

EWLM resource allocation for meeting job goals

We hope this article provided a solid groundwork to make Tivoli Dynamic Workload Broker and Enterprise Workload Manager work together like a seasoned team for more efficient job scheduling and dispatching.

EWLM possesses several resource management capabilities, including logical partition management and provisioning. After work is classified into EWLM service classes, each of the resource management capabilities may be activated to help Tivoli Dynamic Workload Broker work meet its goals.

Resource management capabilities are different for each platform, so please consult the appropriate EWLM documentation if resource management is your goal. Future versions of EWLM may have additional resource management capabilities that can also be employed to meet administratively set goals.

Resources

Learn

• Server/Application State Protocol version 1 (SASPv1) draft: This draft describes how the SASP protocol provides a mechanism for load balancers and workload management systems to communicate better ways of distributing the existing workload to the group members.
  
  
• "Performance monitoring with Enterprise Workload Manager" (developerWorks, March 2006): Learn how to enable ARM instrumentation for end-to-end performance monitoring in IBM middleware on AIX and Linux.
  
  
• "Fit components together to manage resources and workloads" (developerWorks, February 2005): This article explains how to use three products -- EWLM, Tivoli Provisioning Manager, and Tivoli Intelligent Orchestrator (plus ARM and SASP) -- to provide a cohesive infrastructure.
  
  
• developerWorks Tivoli zone: Visit the Tivoli resources for articles, tutorials, news, and other resources on Tivoli technologies.
  
  
• developerWorks technical events and webcasts: Stay current with the latest technology.
  
  

Get products and technologies

• IBM Enterprise Workload Manager SDK: Try this SDK to aid in the development and test of Application Response Measurement 4.0-level instrumentation in applications and middleware, and the development and test of EWLM ARM Adapter library implementations.
  
  
• IBM trial software: Build your next development project with trial software, available for download directly from developerWorks.
  
  

Discuss

• Participate in the discussion forum.
  
  
• developerWorks blogs: Get involved in the developerWorks community.
  
  
• Autonomic computing forums: These forums cover subject areas of interest to the technology.
  
  
• Dave Bartlett blogs: IBM VP Dave Bartlett blogs on autonomic computing issues, as does the zone blog, "Inside Autonomic Computing."
  
  

About the authors

Alan Bivens, PhD, is a research staff member at the IBM T.J. Watson Research Center where he creates autonomic workload management capabilities to allow datacenters to be self-healing and self-optimizing. His current work deals with load balancing, power management, and coordination between autonomic managers.

Tullio Tancredi is an IBM Software Engineer at the Rome Tivoli Laboratory. His previous experience includes working on several projects in the availability and monitoring areas as well as autonomic computing and developing the Tivoli Autonomic Management Engine. Last year he joined the workload and scheduling development area, working on the IBM Tivoli Dynamic Workload Broker project.

Comments

Back to top

Trademarks

Table of contents

• Before you start
• Introducing Tivoli Dynamic Workload Broker
• Introducing EWLM
• Planning the interaction
• EWLM load balancing recommendations in Tivoli Dynamic Workload Broker
• Starting the interaction
• Enabling ARM on the Tivoli Dynamic Workload Broker Agent
• EWLM classification of Tivoli Dynamic Workload Broker jobs
• Confirming the interaction from the EWLM Control Center
• Application level load balancing
• EWLM resource allocation for meeting job goals
• Resources
• About the authors
• Comments

Local resources

• IBM Innovation Center - Austin, TX
• IBM Innovation Center - Chicago, IL
• IBM Innovation Center - Dallas, TX
• IBM Innovation Center - San Mateo, CA
• IBM Innovation Center - Waltham, MA
• Technical Briefings in North America

My developerWorks community

Dig deeper into Tivoli on developerWorks

We want to know you!

Update your My developerWorks profile and start connecting with others

Special offers