How do I change the heap sizes for Elasticsearch and Logstash?

Scale up the default Elasticsearch and Logstash heap size for each service to match your deployment.

About this task

Before deciding on the heap size for your deployment, consider these guidelines:
  • Elasticsearch recommends that you assign 50 percent of available memory to the Elasticsearch client service, but not exceed 30.5 GB. Based on these recommendations, configure the Elasticsearch client and data services heap in IBM® Spectrum Conductor to use 6 to 8 GB. If your hosts have more memory (for example, 1 TB), tune these options as recommended in Heap: Sizing and Swapping.
  • The default garbage collector for Elasticsearch is Concurrent-Mark and Sweep (CMS). To prevent long stop-the-world pauses, do not configure the heap size to be higher than what the CMS garbage collector was designed for (6 to 8 GB). . For more information, see Elasticsearch setup and configuration.

Procedure

Follow these steps to scale up your Elasticsearch and Logstash heap sizes:

  1. Edit the $EGO_CONFDIR/../../integration/elk/conf/elk.conf configuration file.
  2. Locate the Elasticsearch and Logstash heap environment variables. By default, the heap sizes in IBM Spectrum Conductor are as follows:
    ES_CLIENT_HEAP_SIZE=4g
    ES_DATA_HEAP_SIZE=4g
    ES_MASTER_HEAP_SIZE=2g
    LS_HEAP_SIZE=4g
    
  3. Increase the heap sizes.
    To prevent the heap from resizing at run time, the minimum (Xms) and maximum (Xmx) sizes are set to one value; both use the value of the Elasticsearch heap environment variables.
  4. Save your changes.
  5. Restart the Elasticsearch services that you modified. For example, if you modified the Elasticsearch data node service only, restart the elk-elasticsearch-data service:
    egosh service stop elk-elasticsearch-data
    egosh service start elk-elasticsearch-data
    Your new heap size configuration takes effect.