Description

Submits a job for execution and assigns it a unique numerical job ID.

Runs the job on a host that satisfies all requirements of the job, when all conditions on the job, host, queue, application profile, and cluster are satisfied. If LSF cannot run all jobs immediately, LSF scheduling policies determine the order of dispatch. Jobs are started and suspended according to the current system load.

Sets the user's execution environment for the job, including the current working directory, file creation mask, and all environment variables, and sets LSF environment variables before the job starts.

When a job is run, the command line and stdout/stderr buffers are stored in the directory /.lsbatch on the execution host. If this directory is not accessible, /tmp/.lsbtmp is used as the job's home directory. If the current working directory is under the home directory on the submission host, then the current working directory is also set to be the same relative directory under the home directory on the execution host.

By default, if the current working directory is not accessible on the execution host, LSF finds a working direction to run the job in the following order:

$PWD
Strip /tmp_mnt if it is exists in the path
Replace the first component with a key in /etc/auto.master and try each key
Replace the first 2 components with a key in /etc/auto.master and try for each key
Strip the first level of the path and try the rest (for example, if the current working directory is /abc/x/y/z, try to change directory to the path /x/y/z)
/tmp

If the environment variable LSB_EXIT_IF_CWD_NOTEXIST is set to Y and the current working directory is not accessible on the execution host, the job exits with the exit code 2.

If no command is supplied, bsub prompts for the command from the standard input. On UNIX, the input is terminated by entering CTRL-D on a new line. On Windows, the input is terminated by entering CTRL-Z on a new line.

To kill a job that is submitted with bsub, use bkill.

Use bmod to modify jobs that are submitted with bsub. bmod takes similar options to bsub.

Jobs that are submitted to a chunk job queue with the following options are not chunked; they are dispatched individually:

-I (interactive jobs)
-c (jobs with CPU limit greater than 30)
-W (jobs with run limit greater than 30 minutes)

To submit jobs from UNIX to display GUIs through Microsoft Terminal Services on Windows, submit the job with bsub and define the environment variables LSF_LOGON_DESKTOP=1 and LSB_TSJOB=1 on the UNIX host. Use tssub to submit a Terminal Services job from Windows hosts. See Using LSF on Windows for more details.

If the parameter LSB_STDOUT_DIRECT in lsf.conf is set to Y or y, and you use the -o or -oo option, the standard output of a job is written to the file you specify as the job runs. If LSB_STDOUT_DIRECT is not set, and you use -o or -oo, the standard output of a job is written to a temporary file and copied to the specified file after the job finishes. LSB_STDOUT_DIRECT is not supported on Windows.

Default behavior

LSF assumes that uniform user names and user ID spaces exist among all the hosts in the cluster. That is jobs run under on the execution host the same users account as the job was submitted with. For situations where nonuniform user names and user ID spaces exist, account mapping must be used to determine the account that is used to run a job.

bsub uses the command name as the job name. Quotation marks are significant.

Options that are related to file names and job spooling directories support paths that contain up to 4094 characters for UNIX and Linux, or up to 255 characters for Windows.

Options that are related to command names and job names can contain up to 4094 characters for UNIX and Linux, or up to 255 characters for Windows.

Options for the following resource usage limits are specified in KB:

Core limit (-C)
Memory limit (-M)
Stack limit (-S)
Swap limit (-v)

Use LSF_UNIT_FOR_LIMITS in lsf.conf to specify a larger unit for the limit (MB, GB, TB, PB, or EB).

If fair share is defined and you belong to multiple user groups, the job is scheduled under the user group that allows the quickest dispatch.

The job is not checkpoint-able.

bsub automatically selects an appropriate queue. If you defined a default queue list by setting LSB_DEFAULTQUEUE environment variable, the queue is selected from your list. If LSB_DEFAULTQUEUE is not defined, the queue is selected from the system default queue list that is specified by the LSF administrator with the DEFAULT_QUEUE parameter in lsb.params.

LSF tries to obtain resource requirement information for the job from the remote task list that is maintained by the load sharing library. If the job is not listed in the remote task list, the default resource requirement is to run the job on a host or hosts that are of the same host type as the submission host.

bsub assumes that only one processor is requested.

bsub does not start a login shell but runs the job file under the execution environment from which the job was submitted.

The input file for the job is /dev/null (no input).

bsub sends mail to you when the job is done. The default destination is defined by LSB_MAILTO in lsf.conf. The mail message includes the job report, the job output (if any), and the error message (if any).

bsub charges the job to the default project. The default project is the project that you define by setting the environment variable LSB_DEFAULTPROJECT. If you do not set LSB_DEFAULTPROJECT, the default project is the project that is specified by the LSF administrator with DEFAULT_PROJECT parameter in lsb.params. If DEFAULT_PROJECT is not defined, then LSF uses default as the default project name.

Output

If the job is successfully submitted, bsub displays the job ID and the queue that the job was submitted to.

Limitations

When account mapping is used, the command bpeek does not work. File transfer with the -f option to bsub requires rcp to be working between the submission and execution hosts. Use the -N option to request mail, or the -o and -e options to specify an output file and error file.

Submitting jobs with SSH

Secure Shell (SSH) is a network protocol that provides confidentiality and integrity of data using a secure channel between two networked devices.

SSH uses public-key cryptography to authenticate the remote computer and allow the remote computer to authenticate the user, if necessary.

SSH is typically used to log into a remote machine and execute commands, but it also supports tunneling, forwarding arbitrary TCP ports and X11 connections. SSH uses a client-server protocol.

SSH uses private/public key pairs to log into another host. Users no longer have to supply a password every time they log on to a remote host.

SSH is used when running any of the following jobs:

An interactive job (bsub -IS | -ISp | ISs)
An interactive X-window job with X11 forwarding (bsub -XF)
An interactive X-window job, without X11 forwarding (bsub -IX)
An externally submitted job (esub)

SSH only supports UNIX for submission and execution hosts. The display host can be any operating system.

Depending on your requirements for X-Window jobs, you can choose either bsub -XF (recommended) or bsub -IX. Both options encrypt the X-Server and X-Clients.

SSH X11 forwarding (bsub -XF) has the following benefits:

Any password required can be typed in when needed.
Does not require the X-Server host to have the SSH daemon installed.

SSH X11 forwarding has the following drawbacks:

The user must enable X11 forwarding in the client.
Submission and execution hosts must be UNIX hosts.

SSH X11 forwarding has the following dependencies:

OpenSSH 3.9p1 and up is supported.
OpenSSL 0.9.7a and up is supported.
You must have SSH correctly installed on all hosts in the cluster.
You must use an SSH client to log on to the submission host from the display host.
You must install and run the X-Server program on the display host.

SSH X11 forwarding has the following limitations:

You cannot run with bsub -K, -IX, or -r.
You cannot bmod a job submitted with X11 forwarding.
Cannot be used with job arrays, job chunks, or user account mapping.
Jobs submitted with X11 forwarding cannot be checked or modified by esubs.
Can only run on UNIX hosts (submission and execution hosts).

SSH interactive X-Window (bsub -IX) has the following benefits:

The execution host contacts the X-Server host directly (no user steps required).
Hosts can be any OS that OpenSSH supports.

SSH interactive X-Window has the following drawbacks:

Requires the SSH daemon installed on the X-Server host.
Must use private keys with no passwords set.

SSH interactive X-Window has the following dependencies:

You must have OpenSSH correctly installed on all hosts in the cluster.
You must generate public/private key pairs and add the content of the public key to the authorized_keys file on remote hosts. For more information, refer to your SSH documentation.
For X-window jobs, you must set the DISPLAY environment variable to X-serverHost:0.0, where X-serverHost is the name of the X-window server. Ensure that the X-server can access itself. Run, for example, xhost +localhost.

SSH interactive X-Window has the following limitations:

Cannot be used with job arrays or job chunks.
Private user keys must have no password set.
You cannot run with -K , -r, or -XF.