Enable a remote job directory to improve performance
For better performance, you may choose to modify an application and enable the remote job directory, which is the local job directory on the execution host. Once the job creates a job directory locally on the execution host, you can operate on the directory through IBM Spectrum LSF Application Center as if it was a shared data directory.
By default, applications do not support a remote job directory. If you want to enable the feature, the application script must generate a bsub submission script.
In order for you to view and manage the remote job directory though IBM Spectrum LSF Application Center, the submission script must generate the .rspooler file in the job directory. The .rspooler file in the job directory specifies to the remote directories used by the job on execution hosts.
The .rspooler file has one remote job directory per line and can have multiple lines. The file format is:
remote_host_name:/remote_directory
For example:
host1:/tmp/usr1/job1directory
host2:/stratch/dev/shared/job2directoryOnce the .rspooler file is generated in the job directory, if the parameter DELETE_REMOTE_DIR=y is configured in $GUI_CONFDIR/pmc.conf, you will be able to view and delete any remote job directories through IBM Spectrum LSF Application Center, and automatic data purging will also be performed on the remote job directories.
For example, you can edit the ABAQUS submission template and add this code at the end of the script. The bsub submission script creates the remote job directory before starting the job. You can modify other applications based on this example.
##########################################################
# Begin: create bsub submission script
##########################################################
#
BSUB_SCRIPT=$OUTPUT_FILE_LOCATION/bsub.$JOB_NAME
exec 3>&1 # Link file descriptor #3 with stdout.
exec > $BSUB_SCRIPT # stdout replaced with file "bsub.$JOBNAME".
#
echo "#!/bin/sh"
echo "#BSUB $RUNHOST_OPT"
echo "#BSUB $JOB_NAME_OPT"
echo "#BSUB $DECK_OPT"
echo "#BSUB $QUEUE_OPT"
echo "#BSUB $NCPU_OPT"
echo "#BSUB $OUTPUT_OPT"
echo "#BSUB $MEMARC_OPT"
echo "#BSUB $ABQ_LIC"
echo "#BSUB $EXTRA_PARAMS"
#
# Create the remote job directory on the computes nodes for the job.
# The logic below removes duplicates when using hosts with multiple CPUs.
# "/tmp/$EXECUTIONUSER/" is used as remote job directory in the example code below
#
echo " echo \’hostname\’ >> $"OUTPUT_FILE_LOCATION"/nodelist"
echo " echo \’hostname\’:/tmp/$EXECUTIONUSER/$JOB_NAME >> $"OUTPUT_FILE_LOCATION"/.rspooler"
echo " mkdir -p /tmp/$EXECUTIONUSER/$JOB_NAME"
#
# Copy job files to the local scratch directory.
#
echo "cd /tmp/$EXECUTIONUSER/$JOB_NAME"
echo "cp $OUTPUT_FILE_LOCATION/* /tmp/$EXECUTIONUSER/$JOB_NAME"
#
# Create the EXE_CMD for the application used in the bsub submission script
EXE_CMD="${ABAQUS_CMD} ${ABAQUS_CPU_OPT} ${ABAQUS_OPTIONS} ${OTHER_OPTS} int"
echo "$EXE_CMD"
#
exec 1>&3 3>&- # Restore stdout and close file descriptor #3.
##########################################################
# End: create bsub submission script
##########################################################
JOB_RESULT=`/bin/sh -c "bsub < $BSUB_SCRIPT 2>&1"`
export JOB_RESULT OUTPUT_FILE_LOCATION
${GUI_CONFDIR}/application/job-result.sh