How to tune the rate of job processing
The processing of jobs submitted for dynamic scheduling is handled by the two subcomponents of dynamic workload broker, job dispatcher and resource advisor, through a mechanism of queues and a cache memory. Job dispatcher uses a system of queues into which jobs are placed according to their processing status and thus transmitted to the resource advisor. Resource advisor uses a system of time slots during which it takes a number of jobs from the job dispatcher and allocates them to the resources that will run them.
The JobDispatcherConfig.properties and ResourceAdvisorConfig.properties configuration
files are tuned to suit most environments. However, if your environment
requires a high job throughput or if jobs are processed too slowly,
you can add the parameters listed below to the specified configuration
files and provide customized values. The configuration files are created
for dynamic workload broker at
installation time and are documented in IBM® Workload Scheduler: Administration
Guide.
By default, the parameters listed below are not listed in the configuration files to prevent unwanted modifications. Only expert administrators should set these parameters.
After modifying these parameters, stop and restart dynamic workload broker, as explained in https://www.ibm.com/support/knowledgecenter/SSGSPN_9.4.0/com.ibm.tivoli.itws.doc_9.4/distr/src_ref/awsrgstartbrokerapp.htm.
- JobDispatcherConfig.properties
-
- MaxProcessingWorkers
- Job dispatcher queues the submitted jobs according to their processing
status. By default the following 3 queues are already specified:
Queue.actions.0 = cancel, cancelAllocation, completed, cancelOrphanAllocation Queue.actions.1 = execute, reallocateAllocation Queue.size.1 = 20 Queue.actions.2 = submitted, notification, updateFailedEach queue is determined by the keywords:- Queue.actions.queue_number
- Specifies the jobs added in this queue based on their processing
status. The queue_number identifies the queue and ranges from
0 to 9. You can specify a maximum of 10 queues. The following table
shows the entire list of process statuses you can specify in the queues.
Unspecified job processing statuses are automatically placed in queue 0.Table 1. Job processing status to queue jobs for dispatching Job processing statuses: activated cancel cancelAllocation cancelJobCommand cancelOrphanAllocation childActivated childCompleted childDeactivated childStarted completed deleteJobCommand execute getJobLogCommand getJobPropertiesCommand holdJobCommand notification reallocateAllocation reconnect resumeJobCommand submitJobCommand submitted updateFailed - - - Queue.size.queue_number
- Specifies the number of threads available to the queue identified
by queue_number. You can specify 1 to 100 threads for each
queue you define. The default is the number specified for
MaxProcessingWorkers.
MaxProcessingWorkersspecifies the default number of concurrent threads available to each queue. Each job dispatcher queue usesMaxProcessingWorkersthreads, unless otherwise specified inQueue.size.queue_number. TheMaxProcessingWorkersdefault is 10. Of the three default queues shown above, only queue 1 has its size specified to 20 threads (or workers). Queues 0 and 2 use the default defined inMaxProcessingWorkers(10 threads).For example, in a test scenario with 250K jobs submitted through the workload broker workstation, the job allocation queues are re-configured as follows:# Override default settings Queue.actions.0 = cancel, cancelAllocation, cancelOrphanAllocation Queue.size.0 = 10 Queue.actions.1 = reallocateAllocation Queue.size.1 = 10 Queue.actions.2 = updateFailed Queue.size.2 = 10 # Relevant to jobs submitted from # workload broker workstation, when successful Queue.actions.3 = completed Queue.size.3 = 50 Queue.actions.4 = execute Queue.size.4 = 50 Queue.actions.5 = submitted Queue.size.5 = 50 Queue.actions.6 = notification Queue.size.6 = 50 # Default for every queue size MaxProcessingWorkers = 10Tune this parameter carefully to avoid impairing product performance.
- HistoryDataChunk
- Specifies the number of jobs to be processed at the same time when moving job data to the archive database. This is applicable only to a DB2® RDBMS. This parameter prevents an overload on the job dispatcher. The unit of measurement is jobs. The default value is 1000 jobs.
- ResourceAdvisorConfig.properties
-
- MaxAllocsPerTimeSlot
- Specifies the number of requests for job allocation to be processed
for each time slot. The default value is 100 requests per time slot.
By default, each time slot lasts 15 seconds. Increasing this number
causes the resource advisor to process a higher number of resource
allocation requests per time slot with consequent processor time usage.
This also allows the processing of a higher number of jobs per time
slot. Decreasing this number causes the resource advisor to process
a lower number of resource allocation requests per time slot resulting
in a smoother processor usage and slower job submission processing.
You can also modify the time slot duration using the
TimeSlotLengthparameter available in this file. - MaxAllocsInCache
- Specifies the number of requests for job allocation submitted
by job manager to the resource advisor and stored in its cache. This
number should be substantially higher than the value specified in
the
MaxAllocsPerTimeSlotparameter. The default value is 5000 allocation requests. Increasing this number causes the resource advisor to process a potentially higher number of resource reservations per time slot with consequent processor time usage. This also allows the processing of a higher number of jobs. Decreasing this number causes the resource advisor to process a lower number of resource reservations per time slot resulting in lower processor usage and slower job submission processing. For optimal performance, this value should be at least 10 times the value specified in theMaxAllocsPerTimeSlotparameter.