Tuning the heap sizes for Elasticsearch and Logstash to accommodate heavy load

Scale up the default Elasticsearch and Logstash heap size for each service to match your deployment. For example, if you increase memory for your system, you can also increase the heap size to accommodate system performance and heavy workload.

About this task

The default Elasticsearch installation uses 10 GB heap for the Elasticsearch services and 4 GB for Logstash service, which satisfies the 24 GB of RAM for IBM® Spectrum Conductor requirements as described in Installation requirements and considerations. If your hosts have more than 24 GB memory, you can increase the Elasticsearch and Logstash heap sizes in IBM Spectrum Conductor using this topic.
Before deciding the heap size for your deployment, additionally consider these guidelines:

Procedure

Follow these steps to scale up your Elasticsearch and Logstash heap sizes:

  1. Edit the $EGO_CONFDIR/../../integration/elk/conf/elk.conf configuration file.
  2. Locate the Elasticsearch and Logstash heap environment variables. By default, the heap sizes in IBM Spectrum Conductor are as follows:
    ES_CLIENT_HEAP_SIZE=4g
    ES_DATA_HEAP_SIZE=4g
    ES_MASTER_HEAP_SIZE=2g
    LS_HEAP_SIZE=4g
    
  3. Increase the heap sizes by changing the default heap sizes (4g, 4g, 2g, and 4g) to larger values.

    To prevent the heap from resizing at run time, the minimum (Xms) and maximum (Xmx) sizes are set to one value; both use the value of the Elasticsearch heap environment variables.

  4. Save your changes.
  5. Restart the Elasticsearch services that you modified. For example, if you modified the Elasticsearch data node service only, restart the elk-elasticsearch-data service:
    egosh service stop elk-elasticsearch-data
    egosh service start elk-elasticsearch-data
    Your new heap size configuration takes effect.