lsb.modules

The lsb.modules file contains configuration information for LSF scheduler and resource broker modules. The file contains only one section, named PluginModule.

This file is optional. If no scheduler or resource broker modules are configured, LSF uses the default scheduler plug-in modules named schmod_default and schmod_fcfs.

The lsb.modules file is stored in the directory LSB_CONFDIR/cluster_name/configdir. The LSB_CONFDIR parameter is defined in the lsf.conf file.

After you change lsb.modules configuration

After you change the lsb.modules file, run the badmin reconfig command to reconfigure the mbatchd daemon.

PluginModule section

Description

The PluginModule section defines the plug-in modules for the LSF scheduler and LSF resource broker. If the PluginModule section is not configured, LSF uses the default scheduler plug-in modules that are named schmod_default and schmod_fcfs, which enable the LSF default scheduling features.

Example PluginModule section

The following PluginModule section enables all scheduling policies provided by LSF:

Begin PluginModule 
SCH_PLUGIN              RB_PLUGIN           SCH_DISABLE_PHASES
schmod_default               ()                      () 
schmod_fairshare             ()                      () 
schmod_fcfs                  ()                      () 
schmod_limit                 ()                      () 
schmod_parallel              ()                      () 
schmod_reserve               ()                      () 
schmod_preemption            ()                      ()
schmod_advrsv                ()                      () 
schmod_mc                    ()                      () 
schmod_jobweight             ()                      () 
schmod_cpuset                ()                      () 
schmod_pset                  ()                      ()
schmod_ps                    ()                      () 
schmod_aps                   ()                      () 
schmod_affinity              ()                      () 
schmod_demand                ()                      () 
End PluginModule

PluginModule section structure

The first line consists of the following keywords:

SCH_PLUGIN
RB_PLUGIN
SCH_DISABLE_PHASES

They identify the scheduler plug-ins, resource broker plug-ins, and the scheduler phase to be disabled for the plug-ins that you want to configure.

Each line describes the configuration information for one scheduler plug-in module, resource broker plug-in module, and scheduler phase, if any, to be disabled for the plug-in. Each line must contain one entry for each keyword. Use empty parentheses ( ) or a dash (-) to specify the default value for an entry.

SCH_PLUGIN

Description

Required. The SCH_PLUGIN column specifies the shared module name for the LSF scheduler plug-in. Scheduler plug-ins are called in the order they are listed in the PluginModule section.

By default, all shared modules for scheduler plug-ins are located in the directory defined by the LSF_LIBDIR parameter in the lsf.conf file. On UNIX and Linux, you can also specify a full path to the name of the scheduler plug-in.

The following modules are supplied with LSF:

schmod_default

Enables the default LSF scheduler features.

schmod_fcfs

Enables the first-come, first-served (FCFS) scheduler features. schmod_fcfs can appear anywhere in the SCH_PLUGIN list. By default, if schmod_fcfs is not configured in the lsb.modules file, it is loaded automatically along with schmod_default.

Source code (sch.mod.fcfs.c) for the schmod_fcfs scheduler plug-in module is installed in the directory

LSF_TOP/10.1.0/misc/examples/external_plugin/

Use the LSF scheduler plug-in SDK to modify the FCFS scheduler module code to suit the job scheduling requirements of your site.

schmod_fairshare

Enables the LSF fair share scheduling features.

schmod_limit

Enables the LSF resource allocation limit features.

schmod_parallel

Enables scheduling of parallel jobs submitted with bsub -n.

schmod_reserve

Enables the LSF resource reservation features.

To enable processor reservation, backfill, and memory reservation for parallel jobs, you must configure both schmod_parallel and schmod_reserve in lsb.modules. If only schmod_reserve is configured, backfill and memory reservation are enabled only for sequential jobs, and processor reservation is not enabled.

schmod_preemption

Enables the LSF preemption scheduler features.

schmod_advrsv

Handles jobs that use advance reservation (brsvadd, brsvs, brsvdel, bsub -U)

schmod_cpuset

Handles jobs that use SGI cpusets (bsub -ext[sched] "CPUSET[cpuset_options]")

The schmod_cpuset plug-in name must be configured after the standard LSF plug-in names in the PluginModule list.

schmod_mc

Enables IBM® Spectrum LSF multicluster capability job forwarding

schmod_ps

Enables resource ownership functionality of EGO-enabled SLA scheduling policies

schmod_aps

Enables absolute priority scheduling (APS) policies configured by the APS_PRIORITY parameter in the lsb.queues file.

The schmod_aps plug-in name must be configured after the schmod_fairshare plug-in name in the PluginModule list so that the APS value can override the fair share job-ordering decision.

schmod_affinity

Enables configured by the AFFINITY parameter in the lsf.conf file.

schmod_demand

Enables LSF resource connector functionality configured by the LSB_RC_EXTERNAL_HOST_FLAG parameter in the lsf.conf file.

Scheduler plug-in SDK

Use the LSF scheduler plug-in SDK to write customized scheduler modules that give you more flexibility and control over job scheduling. Enable your custom scheduling policies by configuring your modules under SCH_PLUGIN in the PluginModules section of the lsb.modules file.

The following directory contains sample plug-in code:

LSF_TOP/10.1.0/misc/examples/external_plugin/

SCH_DISABLE_PHASES

Description

The SCH_DISABLE_PHASES column lists the scheduler phases, if any, to be disabled for the plug-in. LSF scheduling has four phases:

Preprocessing - the scheduler checks the readiness of the job for scheduling and prepares a list of ready resource seekers. It also checks the start time of a job, and evaluates any job dependencies.
Match/limit - the scheduler evaluates the job resource requirements and prepares candidate hosts for jobs by matching jobs with resources. It also applies resource allocation limits. Jobs with all required resources matched go on to order/allocation phase. Not all jobs are mapped to all potential available resources. Jobs without any matching resources will not go through the Order/Allocation Phase but can go through the Post-processing phase, where preemption might be applied to get resources the job needs to run.
Order/allocation - the scheduler sorts jobs with matched resources and allocates resources for each job, assigning job slot, memory, and other resources to the job. It also checks whether the allocation satisfies all constraints defined in configuration, such as queue slot limit, deadline for the job.
1. In the order phase, the scheduler applies policies such as FCFS, fair share, and host-partition and consider job priorities within user groups and share groups. By default, job priority within a pool of jobs from the same user is based on how long the job has been pending.
2. For resource-intensive jobs (jobs that require many CPUs or a large amount of memory), resources are reserved so that these jobs are not starved.
3. When all the currently available resources are allocated, jobs go on to post-processing.
Post-processing - the scheduler prepares jobs from the order/allocation phase for dispatch. Post-processing also applies preemption or backfill policies to obtain resources for the jobs with preprocessing or match or limit phases that are complete, but which did not have resources available to enter the next scheduling phase.

Each scheduler plug-in module invokes one or more scheduler phases. The processing for a give phase can be disabled or skipped if the plug-in module does not need to do any processing for that phase or the processing is already done by a previous plug-in module in the list.

Default

Undefined