Enabling LSF Elasticsearch functionality

Edit online

To enable Elasticsearch functionality in IBM Spectrum LSF configure the parameters LSF_QUERY_ES_FUNCTIONS and LSF_QUERY_ES_SERVERS in the lsf.conf configuration file. Additional configuration to the config.json and demo.conf files.

LSF_QUERY_ES_FUNCTIONS

Edit online
Tip: For more information about the LSF_QUERY_ES_FUNCTIONS parameter, refer to the IBM Spectrum LSF content for LSF_QUERY_ES_FUNCTIONS.

Syntax

LSF_QUERY_ES_FUNCTIONS="all"

LSF_QUERY_ES_FUNCTIONS="[acct] | [jobs] | [energy] | [gpfsio]"

Description

Enables LSF Elasticsearch functions.

Specify a space-separated list of function names that use the LSF Elasticsearch.

Use this parameter to enable the LSF Elasticsearch functions. To use these functions, you must also define the LSF_QUERY_ES_SERVERS parameter.

Valid values

This parameter supports the following function names:

  • acct: If this function is defined, the bacct command calls elastic search to get the accounting log records.
  • jobs: If this function is defined, the bhist command calls elastic search to get the event log records.
  • all: If this keyword is defined, all the supported functions are enabled.
Two additional values were added in version 10.2.0.6:
  • energy: If this function is defined, the bhosts and bjobs commands use IBM Spectrum LSF Explorer to retrieve energy data.
  • gpfsio: If this function is defined, the bjobs and bacct commands use IBM Spectrum LSF Explorer to retrieve IBM Spectrum Scale job accounting input and output data.
Specifically, the bjobs -l command shows approximate accumulated job disk usage (input and output) data, the bjobs -o "gpfsio" command shows job disk usage (input and output) data, and the bacct -l command shows the total approximate number of read/write bytes of all storage pools, all on IBM Spectrum Scale.

Default

Not defined

LSF_QUERY_ES_SERVERS

Edit online
Tip: For more information about the LSF_QUERY_ES_SERVERS parameter, refer to the IBM Spectrum LSF content for LSF_QUERY_ES_SERVERS.

Syntax

LSF_QUERY_ES_SERVERS="[https://ip_address:port_number | https://host_name:port_number ip_address:port_number | host_name:port_number] ..."

Description

Define a space-separated list of IP and port numbers for IBM Spectrum LSF Explorer Servers. This parameter is used for LSF functions that need to call the LSF Elasticsearch.

Important: To install Elasticsearch with HTTPS enabled, see Installing and configuring with HTTPS enabled Elasticsearch.

Default

Not defined

config.json

Edit online
Four parameters are added into the IBM Spectrum LSF Explorer Server's config.json to limit the records size for LSF Elasticsearch functions:
  • "query.chunk.size.max" : 10
  • "query.bhist.jobs" : 5000
  • "query.bhist.days" : 10
  • "query.bacct.days" : 30

Install ipmitool

Edit online

To enable collection of energy accounting information, you must install and configure the ipmitool.

This requires a product ID and corresponding sensor name for your hardware platforms. Some supported hardware platforms product IDs and corresponding sensor names are listed in the following table.

Table 1. Product ID and corresponding sensor name
Product ID Sensor name
43707 System Power
353 AC Avg Power
215 AC Avg Power
307 Avg Power

If the product ID for your hardware is listed in the table, IBM Spectrum LSF Explorer can collect the energy information when this feature is enabled. If your hardware platform is not listed above, you must input the product ID and sensor name in the demo.conf file. The default location of the demo.conf file is /opt/ibm/lsfsuite/lsf/conf/lsfbeats/energy/conf.

Tip: How to retrieve the product ID and sensor name for your hardware:
  1. Run ipmitool mc info and the product ID is displayed in the output.
  2. Run ipmitool sensor and the avg power related sensor name is displayed in the output.