Configuring Elasticsearch

Setting up Elasticsearch servers

To set up and configure an Elasticsearch server, or a cluster of Elasticsearch servers, refer to the Elasticsearch documentation.

• Set up the number of servers and number of shards to ensure that the cluster performs well under the expected load, and scales well in the future.
• Ensure that replicas are configured, so that downtime of one server does not affect the availability of index shards. Further, this will ensure that if one of the index shards is corrupted, it can be restored from one of its replicas.
• Ensure that the Elasticsearch cluster is secured against any potential threats.
• It is strongly recommended to assign server ports through the ‘transport.tcp.port’ elasticsearch.yml configuration property. Giving a fix port here, and using that port while configuring elasticsearch.properties will ensure that Elasticsearch clients are able to reconnect when the server is restarted.
• Ensure that the network, including firewall and firewall rules, are configured to allow access to the Elasticsearch servers by application servers running Sterling Order Management System Software, agent servers, and integration servers.
• Ensure that the Elasticsearch servers are started.

Notes:

• Elasticsearch servers are also referred to as Elasticsearch server nodes throughout the IBM® Sterling Order Management System Software documentation.
• Elasticsearch automatically assigns a name when the server starts. As the selection of the name is completely random, it is recommended to configure 'node.name' property in the Elasticsearch configuration file for all server nodes to ease manageability. Note that each server node should have a unique value for this property.
• While configuring the Elasticsearch configuration file (elasticsearch.yml), ensure that there are no spaces at the beginning of any configured property.

Configuring the Elasticsearch client

All Elasticsearch-specific client properties are configured through elasticsearch.properties.

To configure the Elasticsearch client:

1. Configure yfs.properties_ysc_ext to enable the Sterling Search Index.
2. Configure the Elasticsearch client through elasticsearch.properties:
   • Configure the ‘cluster.name’ property to match the cluster name provided while configuring the Elasticsearch servers.
   • Configure the ‘discovery.zen.ping.unicast.hosts’ property to provide a list of Elasticsearch server IP addresses and their corresponding ports in the format: host1,host2:port,host3[portX-portY]

     Ensure that no quotes or double quotes are used in the value. If ports are explicitly configured in Elasticsearch server nodes, it is strongly recommended to provide those ports here. This will help ensure that the clients are able to reconnect the servers if the servers are restarted.

   • Configure other properties as needed.

Notes:

• Elasticsearch supports three ways through which clients can interact with the Elasticsearch server nodes to index and search data:
  • Through the Elasticsearch client nodes
  • Through the Elasticsearch transport module
  • Through HTTP using RESTful APIs

  The yfs.elasticsearch.client.type property defines the type of Elasticsearch client that is used for establishing a connection and further communication with the Elasticsearch server. The valid values are NodeClient, TransportClient, and JestClient.

  In Sterling Order Management System Software, this property defaults to NodeClient; that is, the Elasticsearch client interacts with the Elasticsearch server nodes through client nodes to index and search data. These clients behave as actual Elasticsearch nodes, even though they are configured to not hold any data. Therefore, although these ‘client nodes’ are Elasticsearch nodes, they do not take ownership of any search index shard or replica. In elasticsearch.properties, the two properties ‘node.client=true’ and ‘node.data=false’ ensure the same. These two properties must not be modified, and a runtime check ensures this.

  The TransportClient connects remotely to an Elasticsearch cluster using the transport module. Unlike NodeClient, it does not join the Elasticsearch cluster but gets one or more initial transport addresses and communicates with them in a round-robin fashion on each action by simply forwarding requests to a node in the cluster. The TransportClient reads host(s) from the Elasticsearch property yfs.elasticsearch.client.initial.hosts and sniffs other nodes in the cluster internally.

  Both NodeClient and TransportClient are dependant on Apache Lucene JARs, whereas a REST client, which uses JSON over HTTP and invokes RESTful APIs on the Elasticsearch server, provides complete independence from Lucene. That is, in the case of a REST client, the Elasticsearch-based search solutions within Sterling Order Management System Software are no longer dependent on Lucene JARs and, therefore, they can be upgraded independently without any change to Lucene or Lucene-based search features such as catalog search. The JestClient, a Java HTTP Rest client for Elasticsearch, is introduced for this purpose. This also requires one or more initial HTTP host addresses to establish a connection with the Elasticsearch cluster.

  The TransportClient uses port 9300 and the JestClient uses port 9200, by default. Elasticsearch servers, by default, bind with port 9300 and port 9200 for transport and HTTP modes, respectively. If there are instances when the Elasticsearch servers are bound to the next available ports, then it is mandatory to set the port along with the initial HTTP host addresses.

• The property file elasticsearch.properties contains a set of properties that are used by Sterling Order Management System Software. These properties start with the ‘yfs.’ prefix. All other properties are passed, as is, to the internal Elasticsearch client node. Notice that all of these properties, including the ‘cluster.name’ and ‘discovery.zen.ping.unicast.hosts’ properties, are the same as those present in the Elasticsearch server configuration file (elasticsearch.yml) as well. Since all properties present in elasticsearch.properties, and not starting with a ‘yfs.’ prefix, are directly passed to the internal Elasticsearch client node, other properties that Elasticsearch understands can be added to this property file. This allows the flexibility to configure the internal Elasticsearch node later for security or performance considerations. Refer to Elasticsearch documentation and Elasticsearch configuration files for allowed values and their use.
• You cannot specify a node.name in the elasticsearch.properties file because this property file is shared by multiple JVMs (application server / agent server / integration server) and all the client nodes end up having the same name. However, if you have different vendor files for each of these servers, configuring a unique 'node.name' in the corresponding elasticsearch.properties file helps to manage the client nodes easily.