Enabling HTTPS for Elasticsearch

Install Elasticsearch with HTTPS enabled and then install IBM Spectrum LSF Explorer server and nodes.

Procedure

Download the basic version of Elasticsearch 7.x (where is x is greater than 2), unzip, and install it; for example:
https://www.elastic.co/downloads/past-releases/elasticsearch-7-2-1

Configure /etc/hosts to ensure the IP address and node name corresponds with yours.
```
# [node1] /etc/hosts:
192.168.0.2 node1.elastic.test.com node1
192.168.0.3 node2.elastic.test.com node2

# [node2] /etc/hosts:
192.168.0.2 node1.elastic.test.com node1
192.168.0.3 node2.elastic.test.com node2
```

Create SSL certificates on node1, and enable TLS for Elasticsearch:

Modify the variable paths according to the download method and storage location of Elasticsearch:

[root@node1 ~]# ES_HOME=/usr/share/elasticsearch
[root@node1 ~]# ES_PATH_CONF=/usr/share/elasticsearch/config

Create the cert file by replacing node1.elastic.test.com and node2.elastic.test.com to the hostname of your Elasticsearch hosts:
```
[root@node1 ~]# mkdir /opt/cert
[root@node1 ~]# cd /opt/cert
```

Create the instance.yml file:

[root@node1 cert]# vi /opt/cert/instance.yml

instances: 
  - name: 'node1'
    dns: [ 'node1.elastic.test.com' ]
  - name: 'node2'
    dns: [ 'node2.elastic.test.com ' ]

Create the CA:

[root@node1 opt]# cd $ES_HOME
[root@node1 opt]# export JAVA_HOME="${ES_HOME}/jdk/"
[root@node1 elasticsearch]# bin/elasticsearch-certutil cert ca --pem --in /opt/cert/instance.yml --out /opt/cert/certs.zip

Unzip it:

[root@node1 elasticsearch]# cd /opt/cert
[root@node1 cert]# unzip certs.zip -d ./certs

Perform Elasticsearch TLS configuration:

Copy the cert files to the config/certs directory:

[root@node1 ~]# cd $ES_PATH_CONF
[root@node1 config]# pwd
/usr/share/elasticsearch/config
[root@node1 config]# mkdir certs
[root@node1 config]# cp /opt/cert/certs/ca/ca.crt /opt/cert/certs/node1/* certs
[root@node1 config]# ll certs
total 12
-rw-r--r--.1 root elasticsearch 1834 Apr 12 08:47 ca.crt
-rw-r--r--.1 root elasticsearch 1509 Apr 12 08:47 node1.crt
-rw-r--r--.1 root elasticsearch 1679 Apr 12 08:47 node1.key

Configure the elasticsearch.yml file:

[root@node1 config]# vi elasticsearch.yml

Note: Ensure the following parameters applied:

network.host: node1.elastic.test.com  #Ensure this is the same as the dns defined in instance.yml.
xpack.security.enabled: true
xpack.security.http.ssl.enabled: true
xpack.security.transport.ssl.enabled: true
xpack.security.http.ssl.key: certs/node1.key
xpack.security.http.ssl.certificate: certs/node1.crt
xpack.security.http.ssl.certificate_authorities: certs/ca.crt
xpack.security.transport.ssl.key: certs/node1.key
xpack.security.transport.ssl.certificate: certs/node1.crt
xpack.security.transport.ssl.certificate_authorities: certs/ca.crt

(Optional) Repeat steps i. and ii. for other Elasticsearch nodes, ensuring to change node1 to the appropriate node name.

Start Elasticsearch:

[root@node1 elasticsearch]# grep '\[node1\] started'
/var/log/elasticsearch/elasticsearch.log
[o.e.n.Node               ] [node1] started

Configure the Elasticsearch username and password. (This only needs to be run once in the LSF Explorer host):

[root@node1 elasticsearch]# cd $ES_HOME
[root@node1 elasticsearch]# bin/elasticsearch-setup-passwords interactive -u 
"https://node1.elastic.test.com:9200"
Initiating the setup of passwords for reserved users 
elastic,apm_system,kibana,logstash_system,beats_system,remote_monitoring_user.
You will be prompted to enter passwords as the process progresses.
Please confirm that you would like to continue [y/N]y


Enter password for [elastic]:
Reenter password for [elastic]:
Enter password for [apm_system]:
Reenter password for [apm_system]:
Enter password for [kibana]:
Reenter password for [kibana]:
Enter password for [logstash_system]:
Reenter password for [logstash_system]:
Enter password for [beats_system]:
Reenter password for [beats_system]:
Enter password for [remote_monitoring_user]:
Reenter password for [remote_monitoring_user]:
Changed password for user [apm_system]
Changed password for user [kibana]
Changed password for user [logstash_system]
Changed password for user [beats_system]
Changed password for user [remote_monitoring_user]
Changed password for user [elastic]

Access the _cat/nodes API using HTTPS:

[root@node1 elasticsearch]# curl --cacert /opt/cert/certs/ca/ca.crt -u elastic 
'https://node1.elastic.test.com:9200/_cat/nodes?v'
Enter host password for user 'elastic':
ip heap.percent ram.percent cpu load_1m load_5m load_15m node.role master name
192.168.0.2   16    95  10 0.76   0.59     0.38 mdi       *      node1

Install or upgrade the Explorer Server and node:
1. Configure install.config, by ensuring the ES_SSL_CERT_PATH is set:
```
ES_HOST=node1.elastic.test.com #Ensure the hostname is the same as the dns defined in instance.yml.
ES_PORT=9200
ES_AUTHENTICATION_ENABLED=Y

ES_SSL_CERT_PATH=/opt/cert/certs/ca/ca.crt
```
2. Install the Explorer Server and node using the installation scripts:
  
  For example, run ./ExplorerServerInstaller.sh -f install.config

(Optional) If you have Kibana installed, to connect Elasticsearch with SSL enabled, perform the following steps:

Change to the Kibana directory and run the following CLIs to set the Elasticsearch username and password to the kibana-keystore:

ES_USERNAME=<your ES username>
ES_PASSWORD=<your ES password>
printf "y" | ./bin/kibana-keystore create
printf ${ES_USERNAME} | ./bin/kibana-keystore add elasticsearch.username -f -x
printf ${ES_PASSWORD} | ./bin/kibana-keystore add elasticsearch.password -f -x

Ensure the following parameters are applied in the kibana.yml file:

elasticsearch.hosts: ["https://your_elasticsearch_host:port"]
elasticsearch.ssl.certificateAuthorities: /path/to/your/ca.crt