Memory preemption

Configure memory preemption

By default, memory is not be preemptable. To enable memory preemption, specify mem in the value of the PREEMPTABLE_RESOURCES parameter in lsb.params. Then, LSF will preempt on both slots and mem.

Jobs with rusage duration

Users are permitted to submit jobs with rusage duration on memory. However, rusage duration will not take effect on memory when memory preemption is enabled, and LSF will continue to reserve memory for a job while it resides on a host.

OS memory behavior

When a job is suspended, it may continue to occupy physical memory. Unless there is another process on the host that can use the memory, the job may not release memory. If LSF launches another job on the host that can use the memory, the OS should start swapping pages of the suspended job out to disk. LSF does not look at swap space as a criteria for preemption.

When jobs exceed their memory requests

If a low priority job exceeds memory allocation on a host and a high priority job needs that memory allocation, you cannot get that memory allocation back through preemption.

For example, suppose that a host has a total of 8 GB memory. A low priority job is submitted, requesting 4 GB memory. However, once the job starts it uses all 8 GB.

A high priority job is submitted that requests 8 GB. LSF sees that there is no memory free on the host. The preemption module calculates that 4 GB memory can be obtained by preempting the low priority job. This is insufficient for the high priority job, so no preemption occurs.