Pending MultiCluster job limit
The pending MultiCluster job limit determines the maximum number of MultiCluster jobs that can be pending in the queue. The queue rejects jobs from remote clusters when this limit is reached. It does not matter how many MultiCluster jobs are running in the queue, or how many local jobs are running or pending.
By default, the limit is 50 pending MultiCluster jobs.
Configure a pending MultiCluster job limit
Edit IMPT_JOBBKLG in lsb.queues, and specify the maximum number of MultiCluster jobs from remote clusters that can be pending in the queue. This prevents jobs from being over-committed to an execution cluster with limited resources.
If you specify the keyword infinit, the queue will accept an infinite number of jobs.
- Considerations
When you set the limit, consider the following:
Make sure there are enough pending jobs in the queue for LSF to dispatch, in order to make full use of the execution servers. If you use advance reservation, set the limit higher to allow for the pending jobs that are waiting to use a reservation.
Make sure the queue does not fill up with so many MultiCluster jobs that LSF cannot dispatch them all in the near future.
Therefore, estimate your expected job flow and set the limit 50% or 100% higher than the estimate.
Example
Assume that locally submitted jobs do not occupy all the available resources, so you estimate that each processor can schedule and execute 2 MultiCluster jobs per scheduling session. To make full use of the job slots, and make sure the queue never runs out of jobs to dispatch, set the limit at 3 or 4 jobs per processor: if this queue has 20 processors, set the limit to allow 60 or 80 MultiCluster jobs pending. You expect to run about 40 of them immediately, and the remainder only wait for one scheduling cycle.