Improving job submission performance

Edit online

Applies to IBM Spectrum LSF Application Center version 9.1.4.1 and higher. In some cases, when you have many job arrays, you can set the parameter MOVE_FINISHED_SHORT_JOBS=true in the $PERF_CONFDIR/dataloader/commonjobloader.properties file to increase the performance of job submission and loading of submission forms in Definitions > Templates on the Workload tab.

Procedure

  1. Log on to the IBM Spectrum LSF Application Center server as root.
  2. Set the IBM Spectrum LSF Application Center environment.
  3. In $GUI_CONFDIR/pmc.conf, check that the FINISH_JOB_TIME_TO_LIVE parameter is set.
  4. Add the parameter MOVE_FINISHED_SHORT_JOBS=true in the configuration file $PERF_CONFDIR/dataloader/commonjobloader.properties.

    The MOVE_FINISHED_SHORT_JOBS parameter defines whether job array elements are individually moved from the live job database table (LSF_LIVE_JOB) to the historical jobs database table (LSF_FINISH_JOB), or whether job array elements in the job are moved together.

    • When MOVE_FINISHED_SHORT_JOBS=true, the job array element is moved from the LSF_LIVE_JOB database table to the LSF_FINISH_JOB database table when the end time of a finished job array element falls within the time defined by the MOVE_FINISHED_JOBS_AFTER parameter in the $PERF_CONFDIR/dataloader/commonjobloader.properties file. As a result, if there are many job arrays, running array elements will be in the LSF_LIVE_JOB database table and finished array elements in the LSF_FINISH_JOB database table, improving performance.
    • When MOVE_FINISHED_SHORT_JOBS=false, job array elements are not moved from the live job database table (LSF_LIVE_JOB) to the historical jobs database table (LSF_FINISH_JOB) until all job array elements finish and the end time of the job array falls within the time defined by the MOVE_FINISHED_JOBS_AFTER in the $PERF_CONFDIR/dataloader/commonjobloader.properties file.
    Note: Ensure the value of MOVE_FINISHED_JOBS_AFTER in commonjobloader.properties is more than the value of CLEAN_PERIOD in lsb.params. Otherwise, when a user performs a rerun job in a job group, IBM Spectrum LSF Application Center will show a duplicate job group.
  5. Restart IBM Spectrum LSF Application Center services. 
    perfadmin start all 
pmcadmin start
  6. Wait one day so that data is moved from the LSF_LIVE_JOB database table to the LSF_FINISH_JOB and LSF_FINISH_JOB_HOST database tables.
  7. Go into IBM Spectrum LSF Application Center, submit jobs, change filter options to display jobs that are older than one day, and check that historical jobs are displayed.