Configuring external Elasticsearch installations

In order to in order to allow Explorer to collect the necessary data (from Elasticsearch) parameters and environment variables must be configured before IBM Spectrum LSF Suite for Workgroups installation.

In lsf-config.yml:

ES_SERVERS: Define the list of the Elasticsearch servers
ES_SSL_CERTIFICATE_PATH: The ssl certificate file path when the protocol Elasticsearch (scheme of the URL) is https
LOGSTASH_TOP: The Logstash top directory accessible from a host with the GUI_Hosts role

If Elasticsearch AUTH is enabled, environment variables must be set on the deployer machine before running lsf-deploy.yml, lsf-upgrade.yml, lsf-edition-upgrade.yml, lsf-apply-ifix.yml, and rollback-ifix.yml. For example:

export ES_USERNAME=elastic
export ES_PASSWORD=password
export ES_JAVACMD=/usr/bin/java (required only if LOGSTASH_TOP is defined)

Perform the following configurations for Elasticsearch, according to your scenario.

Performing a fresh installation of Elasticsearch

ES_SERVERS is defined in lsf-config.yml with an external Elasticsearch installation:
- Suite deployment continues if the ES_SERVERS defined is valid (that is, the external Elasticsearch is ready and running).
- Suite deployment will fail if it fails to contact the external Elasticsearch or the version of the Elasticsearch is lower than 7.2.
When AUTH is enabled in the external Elasticsearch, ES_USERNAME and ES_PASSWORD must be set as environment variables before running lsf-deploy.yml.
If HTTPS is enabled in the external Elasticsearch:
- ES_SSL_CERTIFICATE_PATH must be defined.
- ES_USERNAME, ES_PASSWORD, and ES_JAVACMD must be set as environment variables.
- ES_JAVACMD is used for a Logstash keystore and must point to a non-IBM java on a host with the GUI_Hosts role. It must be a uniform file path for all hosts with a valid LOGSTASH_TOP.
If energy accounting, gpfsio monitoring service, or file beat/metric beat services are required LOGSTASH_TOP must be defined.
- The file path must be accessible from a host with the GUI_Hosts role. If the Logstash path.settings is not LOGSTASH_TOP/config you must make a symbolic link for LOGSTASH_TOP/config to the Logstash path.settings directory. For example:
```
ln -s /etc/logstash LOGSTASH_TOP/config
```
  The Logstash service must be restarted manually to reflect the changes after deploying or upgrading IBM Spectrum LSF Suite for Workgroups.

Enabling HTTPS for Elasticsearch

To enable HTTPS for Elasticsearch, complete the appropriate steps in Enabling HTTPS for Elasticsearch.

Upgrading from IBM Spectrum LSF Suite for Workgroups 10.2 Fix Pack 9 to 10.2 Fix Pack 10

ES_SERVERS will be set with the host names in GUI_Hosts by default (at bin file installation time) to help kick start using the previously bundled Elasticsearch. However it does not mean the previously bundled Elasticsearch is maintained.
If ES_SERVERS is defined with an external Elasticsearch, follow the installation procedure in Installing and configuring Elasticsearch, Kibana, and Logstash with Explorer 10.2 Fix Pack 10 with manual steps to migrate data before running lsf-upgrade.yml.
If the Elasticsearch is not the previously bundled version, stop/remove the bundled ELK services on all GUI hosts before running lsf-upgrade.yml. For example:
```
systemctl stop elasticsearch-for-lsf
systemctl disable elasticsearch-for-lsf
systemctl stop logstash-for-lsf
systemctl disable logstash-for-lsf
```
When AUTH is enabled in the external Elasticsearch, ES_USERNAME and ES_PASSWORD must be set as environment variables before running lsf-deploy.yml.
When AUTH is enabled in the external Elasticsearch, and LOGSTASH_TOP is defined, the following environment variables must be set:
- ES_USERNAME
- ES_PASSWORD
- ES_JAVACMD

Upgrading from IBM Spectrum LSF Suite for Workgroups 10.2 Fix Pack 6, 7, or 8 to 10.2 Fix Pack 10

The requirements for this scenario are the same as those in "Upgrading from IBM Spectrum LSF Suite for Workgroups 10.2 Fix Pack 9 to 10.2 Fix Pack 10".

Follow the installation procedure in Installing and configuring Elasticsearch, Kibana, and Logstash with Explorer 10.2 Fix Pack 10 with manual steps to migrate data before running lsf-upgrade.yml.