Configuring LSF to run Singularity jobs
Configure a Singularity application profile or queue in LSF to run Singularity container jobs.
Before you begin
- Singularity, Version 2.2, or later, must be installed on an LSF server host.
- Add the path to the Singularity executable file to the $PATH environment variable before submitting jobs, and ensure that it is accessible on each execution host.
- The required Singularity images that you intend to use must be prepared before running LSF jobs in containers. Ensure that these images are accessible to each execution host.
- If you use mounted file systems for LSF, other software installations, or for input/output directories, you must also bind required directories into containers to ensure that LSF works properly. To bind directories for Singularity containers, add mount hostfs = yes to the singularity.conf file. This parameter definition binds all mounted file systems to the container.
- To run jobs with bsub -L login_shell or where the execution user is changed by user mapping, the execution user must be able to execute the singularity command under its $PATH environment variable. You can install the singularity binary file in the /usr/bin directory or add a symbolic link to the installation path of singularity.
About this task
- To prepare data for the container as a pre-execution or post-execution operation, put this data into a directory that is mounted to a job container.
- To customize the internal job container, you can customize the starter scripts to prepare the appropriate environment.
If this parameter is specified in both files, the parameter value in the lsb.applications file overrides the value in the lsb.queues file.
- Required. This keyword specifies the Singularity image name that is used in running jobs.
- Optional. This keyword specifies the Singularity job run options, which are passed to the job
container with the singularity exec command in LSF.
To enable a pre-execution script to run, specify an at sign (@) and a full file path to the script, which the execution host must be able to access. Before the container job runs, LSF runs this script with LSF administrator privileges. While the script is running, the jobs' environment variables are passed to the script. When the script finishes running, the output is used as container startup options. The script must provide this output on one line.Note:
- Run singularity exec --help in the command line to view the options that the singularity command supports.
- Before you specify the Singularity job run options, make sure that these options work with the singularity exec command in the command line.
- The $LD_LIBRARY_PATH directory is cleaned according to the setuid bit that Singularity uses to work. Therefore, for programs that depend on $LD_LIBRARY_PATH to work (such as openmpi), ensure that you add $LD_LIBRARY_PATH to the ld.so.conf file and run ldconfig.
Begin Application NAME = singapp CONTAINER = singularity[image(/file/path/ubuntu.img)] DESCRIPTION = Singularity User Service End Application
Begin Application NAME = singappoptions CONTAINER = singularity[image(/file/path/ubuntu.img) options(@/share/usr/sing-options.sh)] DESCRIPTION = Singularity User Service with pre-execution script for options End Application