Use LSF Explorer to improve the performance of the bacct and bhist commands, or to retrieve additional data

Use LSF Explorer to get information for the bacct and bhist commands more efficiently, or to retrieve additional data for LSF Explorer energy accounting or IBM Spectrum Scale I/O accounting.

You can improve the performance of the bacct and bhist commands by using IBM Spectrum LSF Explorer ("LSF Explorer") to get information instead of parsing the lsb.acct and lsb.events files. Using LSF Explorer improves the performance of the bacct and bhist commands by avoiding the need for parsing large log files whenever you run these commands.

You can also integrate LSF Explorer energy accounting into LSF. LSF uses LSF Explorer to collect energy data from each host by the LSF Beats services (lsfbeat tool). The bjobs and bhosts commands can then query the data from LSF Explorer and display the job level or host level energy to users.

If you are using IBM Spectrum Scale as the file system with LSF, you can integrate IBM Spectrum Scale job I/O accounting into LSF. LSF uses LSF Explorer to collect IBM Spectrum Scale process-level I/O consumption and combines this LSF job information to provide job-level IBM Spectrum Scale I/O statistics. This allows you to use LSF commands to show job level IBM Spectrum Scale disk consumption or to use LSF Explorer reports to provide job I/O statistics over time.

To use LSF Explorer to retrieve data, you must have LSF Explorer, Version 10.2, or later, installed and working.

To enable this improvement, edit the lsf.conf file to define the LSF_QUERY_ES_SERVERS and LSF_QUERY_ES_FUNCTIONS parameters:
  1. Set LSF_QUERY_ES_SERVERS="[http://ip:port|http://host_name:port]..."

    If you have HTTPS enabled for Elasticsearch, then specifyLSF_QUERY_ES_SERVERS="[https://ip:port|https://ip:port]..."

    For the ip and port values, provide a space-separated list of IP addresses and port numbers of the LSF Explorer servers that retrieve log records.

  2. Set LSF_QUERY_ES_FUNCTIONS with either of the following syntax:
    • LSF_QUERY_ES_FUNCTIONS="[acct]|[jobs]|[energy]|[gpfsio]
    • LSF_QUERY_ES_FUNCTIONS="all"
    Specify the functions that are permitted to use LSF Explorer to retrieve data:
    acct
    If the acct function is defined, the bacct command uses LSF Explorer to retrieve the accounting log records.
    jobs
    If the jobs function is defined, the bhist command uses LSF Explorer to retrieve the event log records.
    energy
    If the energy function is defined and LSF_ENABLE_BEAT_SERVICE="energy" is defined in the lsf.conf file, the bhosts and bjobs commands use LSF Explorer to retrieve energy data. For more details, refer to LSF_ENABLE_BEAT_SERVICE.
    gpfsio
    If the gpfsio function is defined, the bjobs and bacct commands use LSF Explorer to retrieve IBM Spectrum Scale job accounting I/O data.

    Specifically, the bjobs -l command shows approximate accumulated job disk usage (I/O) data, the bjobs -o "gpfsio" command shows job disk usage (I/O) data, and the bacct -l command shows the total approximate number of read/write bytes of all storage pools, all on IBM Spectrum Scale

    all
    If the all keyword is defined, all supported functions use LSF Explorer to retrieve data.
Note:
  • The bhist -cname, -t, and -T options always bypass LSF Explorer and get the information from the lsb.events file.
  • Use the bacct -f or bhist -f options to manually bypass LSF Explorer and instead use the specified log files. Alternatively, use the bacct -f - or bhist -f - options to manually bypass LSF Explorer and use the lsb.acct or lsb.events log file, respectively.