Memory preemption

Configure memory preemption

By default, memory is not be preemptable. To enable memory preemption, specify the mem keyword in the value of the PREEMPTABLE_RESOURCES parameter in the lsb.params file. LSF preempts on both slots and memory.

Jobs with rusage duration

Users are permitted to submit jobs with rusage duration on memory. However, rusage duration does not take effect on memory when memory preemption is enabled. LSF continues to reserve memory for a job while it resides on a host.

OS memory behavior

When a job is suspended, it might continue to occupy physical memory. Unless there is another process on the host that can use the memory, the job might not release memory. If LSF launches another job on the host that can use the memory, the OS can start swapping pages of the suspended job out to disk. LSF does not look at swap space as a criterion for preemption.

When jobs exceed their memory requests

If a low priority job exceeds memory allocation on a host and a high priority job needs that memory allocation, you cannot get the memory allocation back through preemption.

For example, suppose that a host has a total of 8 GB of memory. A low priority job is submitted, requesting 4 GB of memory. However, once the job starts it uses all 8 GB of memory.

A high priority job is submitted that requests 8 GB of memory. LSF sees that there is no memory free on the host. The preemption module calculates that 4 GB or memory can be obtained by preempting the low priority job. This is not sufficient for the high priority job, so no preemption occurs.