Configuring Elasticsearch disk usage

You can configure Elasticsearch to control disk-based allocation for Elasticsearch to consider available disk space on a node before you determine whether to allocate new shards to that node or to actively relocate shards away from that node.

By default, the cluster.routing.allocation.disk.threshold_enabled configuration is set to true to enable shard allocation based on disk usage and the cluster.routing.allocation.disk.watermark.low watermark is set to 85 percent that controls the low watermark for disk usage. Preventing Elasticsearch from allocating new shards to a host, when disk usage on the host exceeds 85 percent. For more information on these parameters, see the Elastic documentation: Disk-based shard allocation settings.
Follow these steps to modify Elasticsearch disk-based allocation:
Option Steps
Modify settings in the configuration file to manage shard allocation based on disk usage.
  1. In the $EGO_CONFDIR/../../integration/elk/conf/elasticsearch/elasticsearch.yml configuration file, configure the cluster.routing.allocation.disk.watermark.low and cluster.routing.allocation.disk.watermark.high parameters.

    For example, you might updating the low watermark to at least 100 gigabytes free (cluster.routing.allocation.disk.watermark.low": "100gb"), a high watermark of at least 50 gigabytes free ("cluster.routing.allocation.disk.watermark.high": "50gb"), and a flood stage watermark of 10 gigabytes free ("cluster.routing.allocation.disk.watermark.flood_stage": "10gb"), and update the information about the cluster every minute ("cluster.info.update.interval": "1m"):

  2. Run the following command with the value you specified in step 1 to update the settings in the cluster immediately: 
    curl --cacert $EGO_TOP/wlp/usr/shared/resources/security/cacert.pem -u $CLUSTERADMIN:$CLUSTERADMINPASS -H'Content-Type: application/json' -XPUT $es_protocol://$es_hostname:$es_port/_cluster/settings -d '{"transient": {"cluster.routing.allocation.disk.watermark.low": "100gb","cluster.routing.allocation.disk.watermark.high": "50gb","cluster.routing.allocation.disk.watermark.flood_stage": "10gb","cluster.info.update.interval": "1m"}}'
Disable shard allocation based on disk usage.
  1. In the $EGO_CONFDIR/../../integration/elk/conf/elasticsearch/elasticsearch.yml configuration file, set the cluster.routing.allocation.disk.threshold_enabled parameter to false:
    cluster.routing.allocation.disk.threshold_enabled: false
  2. Run the following command to update the settings in the cluster immediately: 
    curl --cacert $EGO_TOP/wlp/usr/shared/resources/security/cacert.pem -u $CLUSTERADMIN:$CLUSTERADMINPASS -H'Content-Type: application/json' -XPUT $es_protocol://$es_hostname:$es_port/_cluster/settings -d '{"transient":{"cluster.routing.allocation.disk.threshold_enabled":false}}'