Staging data out after job completion

The bstage out subcommand copies data files from the job's execution environment to a remote destination at the end of your job script, or in the post-execution script.

About this task

Use the bstage out subcommand to copy data files from the job execution environment to a location created for the job in the staging area cache. The LSF data manager is notified of the successful file copy and submits a transfer job to LSF to copy the file from the cache to the remote location.

After the file is successfully copied from the job execution environment to the cache, you can monitor the status of the output transfer by using the bdata cache command.

Procedure

After a job completes (and generates an output file), copy the file onto the staging area cache by running the bstage out subcommand.
bstage out -src file_path -dst host_name:/file_path

After the file is copied to the staging area cache, the command returns a successful exit code, which notifies the LSF data manager. The LSF data manager submits a transfer job to run on a data transfer node. The transfer node uses the configured data transfer command to copy the file from the cache to the remote location as specified in the -dst option.

  • You can specify only regular files in the –src option. Directories and symbolic links are not permitted.
  • The file path that is specified in the –dst option is passed directly to the transfer command's destination parameter and LSF does not control its behavior.
  • If you do not specify the –dst option, LSF uses the job's submission directory on the submission host by default.
For example, if your job generates an output file /tmp/output.dat on the execution host's local disk, run the following command in your job script to stage out the file:
bstage out –src /tmp/output.dat –dst hostA:/home/outdir