Platform LSF accounting file management

LSF uses accounting files to track resource allocation and usage for all finished jobs. This is the primary purpose of the lsb.acct file. This accounting file is a plain text file with one job record per line. Each job record includes job submission options, allocation information, start time, end time and resource usage. By default, there is only one lsb.acct accounting file per cluster, located in the LSF working directory, which is defined by ${LSB_SHAREDIR}/<cluster_name>/logdir.

LSF logs all finished job records to lsb.acct. By default, no automatic file archiving or rollover mechanism enabled, The lsb.acct file can grow very large, taking up space in the LSF working directory and affecting the performance of commands that query the accounting file (bacct, for instance).

LSF offers several features to help manage account files automatically, including automatic accounting file archiving and automatic deletion of old accounting files.

Automatic accounting file archiving

Automatic accounting file archiving can be triggered by several conditions: time of day, size of the accounting file, and age of accounting files. All of these are configured through parameters in lsb.params.

ACCT_ARCHIVE_AGE and ACCT_ARCHIVE_TIME are mutually exclusive, since ACCT_ARCHIVE_AGE is configured in days, while ACCT_ARCHIVE_TIME sets a specific time of day. If ACCT_ARCHIVE_TIME is defined, ACCT_ARCHIVE_AGE is ignored. ACCT_ARCHIVE_SIZE can coexist with the other two parameters. Run badmin reconfig after you change any of the parameters.

When LSF archives the accounting file, it saves the old accounting file as lsb.acct.n, where n is an index number of archived files. The bigger the index number, the older the archived file. lsb.acct.1 always represents the latest archived accounting file.

Automatic deletion of old accounting files

Configure MAX_ACCT_ARCHIVE_FILE in lsb.params to enable automatic deletion of archived accounting files. Automatic file deletion controls the total number of archived files, and prevents LSF from archiving accounting files indefinitely. MAX_ACCT_ARCHIVE_FILE defines the maximum number of archived files under the LSF working directory. When the total number of archived files reaches the configured threshold, LSF automatically removes the oldest archived file. With this feature, you can control the total number of accounting archives stored under the working directory and make sure accounting files don't consume too much disk space.

Guidelines and best practices for automatic archiving

In a real production environment, you need to choose parameters and values based on the specific needs of your site. Here are some general guidance and best practices: