Known issues, limitations, and troubleshooting

Details of the IBM Spectrum LSF Explorer known issues and limitations for releases 10.1.0 and 10.2.0.

Known issues

The following table describes the known issues in IBM Spectrum LSF Explorer.
Issue	Description
Applicable to 10.1 Fix Pack 3 and later
Calculated field management does not support validation of formula syntax.	When a user saves an invalid formula (which cannot be parsed by Elasticsearch), previewing the affected chart in chart edit mode or opening the affected chart shows nothing on the chart (even after a long processing time).There is no error on the chart for the user to indicate a problem. However, an error (script_execution) is recorded in the Explorer Server log and in the Elasticsearch log. Users must correct or remove the invalid formula.
Applicable to 10.2 and later
User cannot generate snapshot thumbnail for some Dashboards after closing a dashboard	For example, when a user opens a Workload Accounting dashboard, then closes the dashboard, the icon cannot be refreshed by a live snapshot. After several minutes, the icon is replaced by a snapshot of the closed dashboard. However, opening the Cluster Overview dashboard, then closing it, a user can see the new snapshot icon immediately.
When removing a previous IBM Spectrum LSF Explorer Server and Elasticsearch installation, then re-installing the new IBM Spectrum LSF Explorer Server and Elasticsearch, the user may encounter the following issues caused by missing Elasticsearch templates if the previous IBM Spectrum LSF Explorer Data Collectors are not uninstalled and continue to load data: Report open failed with Error - Error 403: Unexpected end of file from server Elasticsearch-based LSF bhist/bacct command failure with error	To resolve missing templates after uninstalling IBM Spectrum LSF Explorer Server and Elasticsearch and re-installing, you need to re-run the init template script installed with IBM Spectrum LSF Explorer Data Collector and clean up any dirty data. Workaround: Logon on to IBM Spectrum LSF Explorer Data Collector host and run the script COLLECTOR_TOP/lsfsuite/ext/perf/1.2/bin/es/initPaTemplate.sh Delete any dirty data in Elasticsearch loaded by IBM Spectrum LSF Explorer Data Collector during IBM Spectrum LSF Explorer Server re-installation: `curl -XDELETE http://[ES_SERVER]:[ES_SERVER_IP]/lsf_* curl -XDELETE http://[ES_SERVER]:[ES_SERVER_IP]/flexlm_*`
The Explorer installation fails and uninstallation script fails.	Installation fails with the following errors: `Preparing... ################################# [100%] package ibm-jre-1.8.0-17092816.x86_64 is already installed Preparing... ################################# [100%] package lsf-perf-10.2.0.0-17092816.x86_64 is already installed Error: Failed to install IBM Spectrum LSF Explorer Data Collector due to the IBM Spectrum LSF Perf installation failed. Exiting ...` Uninstallation fails with the following errors: `[root@adminuser explorer10.2_node_linux-x64]# ./ExplorerNodeUninstall.sh Error: IBM Spectrum LSF Explorer Data Collector is not installed. Exiting ...` The workaround is to manually uninstall the RPMs: `[root@adminuser explorer10.2_node_linux-x64]# rpm -e lsf-perf` Installation of Explorer will then be successful. Use `rpm -qa \|grep <RPM_NAME>` to query whether it was installed.
Enabling the `lsfacctloader` and `lsfeventsloader` data loaders at the same time is not supported.	When the `lsfacctloader` and `lsfeventsloade` data loaders are enabled in parallel, data from the Finished Job Statistics chart will be doubled.
Applicable to 10.2 Fix Pack 6 and later
Server installation fails on Ubuntu 16.04 LTS, Power when the user's home has "rpmdb".	When installing the IBM Spectrum LSF Explorer Server package on Power with Ubuntu 16.04 (`sudo ./ExplorerServerInstaller.sh -f install.config`), the installer is not able to open packages and returns a series of errors: `error: db5 error(-30969) from dbenv->open: BDB0091 DB_VERSION_MISMATCH: Database environment version mismatch error: cannot open Packages index using db5 - (-30969) error: cannot open Packages database in /home/user/.rpmdb Error: Failed to install IBM Spectrum LSF Explorer Server due to the IBM JRE installation failed. Exiting ...` This happens when rpmdb is in the user's home directory. After removing rpmdb from the user's home, the installer will succeed. `sudo rm -rf /home/user/.rpmdb ls -ld /home/user/.rpmdb ls: cannot access '/home/user/.rpmdb': No such file or directory sudo ./ExplorerServerInstaller.sh -f install.config`
Calculated field management does not support validation of formula syntax	When a user saves an invalid formula (which cannot be parsed by Elasticsearch), previewing the affected chart in chart edit mode or opening the affected chart shows nothing on the chart (even after a long processing time).There is no error on the chart for the user to indicate a problem. However, an error (script_execution) is recorded in the IBM Spectrum LSF Explorer Server log and in the Elasticsearch log. Workaround: User must correct or remove the invalid formula.
Applicable to 10.2 Fix Pack 7 and later
When installing IBM Spectrum LSF Application Center with IBM Spectrum LSF Explorer, perf services must be restarted.	When the IBM Spectrum LSF Explorer server has been installed and then IBM Spectrum LSF Application Center is installed in same location, perf services must be restarted: Run `perfadmin stop all`, then `perfadmin start all`. Restart pmc services by running `pmcadmin stop`, then `pmcadmin start`
Applicable to 10.2 Fix Pack 9 and later
Explorer node (data collector) does not uninstall when integrated with Spectrum LSF Application Center.	This issue occurs because perf is uninstalled when Application Center is uninstalled. Therefore, when Application Center and Explorer are both used in the same cluster, if Application Center is uninstalled, then Explorer server is uninstalled, the Explorer node (data collector) uninstall fails: `./ExplorerNodeUninstall.sh Error: IBM Spectrum LSF Explorer Data Collector is not installed. Exiting ...` To work around this issue, manually remove rpms as below: `rpm -e gpfsio-collector-for-lsf-10.2.0.9-19102917.x86_64 rpm -e lsf-beat-10.2.0.6-19102917.x86_64 rpm -e logstash-for-lsf-7.2.1-19102917.noarch cd /opt/ibm rm -rf elastic`
Applicable to 10.2 Fix Pack 10 and later
The Login page is inaccessible on Edge and Chrome browsers when HTTPS is enabled	The user receives a message in the browser such as: `Your connection isn't private. Attackers might be trying to steal your information from ip.nu.mb.er (for example, passwords, messages, or credit cards).NET::ERR_CERT_INVALID`

Limitations

Line, bar, and area charts and graphs are limited to 15 colors.
After data has been inserted in Elasticsearch, the data sampling interval for lsfbhostsloader cannot be changed. Changing the Interval parameter in the file $PERF_TOP/conf/plc/plc_lsf.xml for the lsfbhostsloader data loader(<DataLoader Name="lsfbhostsloader" Interval="600" …) causes the sampling interval to incorrectly become the average of the previously configured value and the new value.
The data sampling interval cannot be changed for the flexlicusageloader data loader. The value is 5 minutes.
In version 10.2 Fix Pack 9, the default cut-off time for Report Editor is the hour of chart creation (not 00:00).
In version 10.2 Fix Pack 9, there may be performance issues for newly introduced box and whisker plots (for CPU utilization and Slot utilization) if the data volume is too large.
The root user cannot log in to IBM Spectrum LSF Explorer due to security implications.

Troubleshooting

Edit online

Cannot open report with unclear message Error 500: Internal Server Error

Sometimes, when a user opens an IBM Spectrum LSF Explorer dashboard or report, there will a log message Error - Error 500: Internal Server Error. This could be caused by following reasons:

Explorer server is down due to port being occupied by another application
Elasticsearch is down
Elasticsearch template or index is not created

How to resolve these issues:

Run pmcadmin status to check whether IBM Spectrum LSF Explorer services have started successfully. If IBM Spectrum LSF Explorer server service hasn't started, check IBM Spectrum LSF Explorer server logs from SERVER_TOP/logs, open server.runtime.log, server.log, to see if there is any error. For example, if IBM Spectrum LSF Explorer server is down due to port occupied by other application, in server.runtime.log there will be an error as follows:
```
Server is exiting ...
events.js:160
throw er; // Unhandled 'error' event
^
Error: listen EADDRINUSE :::5000
```
To resolve the issue, change the IBM Spectrum LSF Explorer server port in SERVER_TOP/config/config.json, edit property: "pa.net.port". Here SERVER_TOP is the top level IBM Spectrum LSF Explorer server location (for example: /opt/ibm/lsfsuite/ext/explorer/server)
If Elasticsearch is down check the IBM Spectrum LSF Explorer server logs from SERVER_TOP/logs, open server.runtime.log, server.log, to see if there is any error. If Elasticsearch is down, there will be error below in server.runtime.log:
```
Elasticsearch ERROR: 2017-11-01T14:54:56Z
Error: Request error, retrying
HEAD http://172.29.2.116:9200/pa-lite-log => connect ECONNREFUSED 172.29.2.116:9200
```
To resolve the issue, start Elasticsearch:
1. Check Elasticsearch service status:
```
 # service elasticsearch-for-lsf status
```
2. Start Elasticsearch service:
```
# service elasticsearch-for-lsf start
 # service elasticsearch-for-lsf status
```
Elasticsearch template or index is not created.
Check the IBM Spectrum LSF Explorer server logs from SERVER_TOP/logs, open server.runtime.log or server.log, to see if there is any error.
If Elasticsearch template or index is not created, in server.log, there will be error like "success":false,"error":"\"visualizations with id [perf_hw01] not found.\""
To resolve the issue, recreate the template and indices:
1. Logon to IBM Spectrum LSF Explorer Server host, navigate to SERVER_TOP/model, and execute the command:
  ./all.sh
  
  Note: SERVER_TOP is the top level IBM Spectrum LSF Explorer Server location (for example: /opt/ibm/lsfsuite/ext/explorer/server)
2. Logon to IBM Spectrum LSF Explorer Data Collector host, navigate to COLLECTOR_TOP/1.2/bin/es, and execute the command:
  ./initPaTemplate.sh
  
  Note: COLLECTOR_TOP is the top level IBM Spectrum LSF Explorer Data Collector location (for example: /opt/ibm/lsfsuite/ext/perf)