Container support

You can now configure Docker, NVIDIA Docker, Shifter, and Singularity containers at the queue level (in addition to the application profile level) to run container jobs.

To enable containers at the queue level, configure the CONTAINER and EXEC_DRIVER parameters in the lsb.queues file in the same manner as you would specify a container application profile in the lsb.applications file.

When scheduling Docker-based containerized jobs, you can now enable LSF to give preference for execution hosts that already have the requested Docker image. This reduces network bandwidth and the job start time because the execution host does not have to pull the Docker image from the repository and the job can immediately start on the execution host.

Enable or disable this feature by specifying the DOCKER_IMAGE_AFFINITY parameter in the lsb.applications, lsb.queues, or lsb.params file. You can also enable or disable this feature at the job level by specifying the LSB_DOCKER_IMAGE_AFFINITY environment variable. The job level setting overrides the application level setting, which overrides the queue level setting, which overrides the cluster level setting.

When this feature is enabled, LSF considers Docker image location information when scheduling Docker jobs. Docker image affinity interacts with host preference and order[] string requests in the following manner:

• If host preference is specified, the host preference is honored first. Among hosts with the same preference level, hosts with the requested Docker image are given higher priority.
• If the order[] string is specified, the hosts with the requested Docker image have a higher priority first. Among hosts that all have the requested Docker image, the order[] string is then honored.