Connecting to an Elasticsearch

Edit online

This section explains the changes that you must make in the config.properties file to enable webMethods API Gateway to communicate with Elasticsearch.

You can also connect to Elasticsearch through externalized configurations. For more details, see Externalizing Configurations.

When
Important: When you use Elasticsearch, you must use the Elasticsearch plugin mapper size. Without this plugin, webMethods API Gateway does not start. To install the mapper size plugin, use the command sudo bin/elasticsearch-plugin install mapper-size. If you have deployed a clustered environment and have configured Elasticsearch on multiple nodes, you must install this plugin on all the nodes that use Elasticsearch.

To connect to Elasticsearch

  1. Navigate to InstallDir/IntegrationServer/packages/WmAPIGateway/config/resources/elasticsearch/config.properties

    The config.properties file contains all the properties and Elasticsearch configurations.

    Note: To exclusively route analytics data to a separate Elasticsearch instance, you need to configure the required properties in the file InstallDir/IntegrationServer/packages/WmAPIGateway/config/resources/elasticsearch/analyticsds.properties
  2. Configure the following properties:
    Property and Description
    pg.gateway.elasticsearch.client.http.response.size

    This property specifies the response size, in MB, for webMethods API Gateway Elasticsearch client.

    Default value: 100

    pg.gateway.elasticsearch.hosts

    Mandatory

    This property lists Elasticsearch hosts and ports. The values are comma separated.

    For example: localhost:9240

    Note: Once a host is added to this property, this is the value that is used to connect to Elasticsearch and the host configured in gateway-es-store.xml is not considered.
    pg.gateway.elasticsearch.http.keepAlive

    Mandatory

    This property creates the persistent connection between client and server.

    Default value: true

    pg.gateway.elasticsearch.http.connectionTimeout

    Mandatory

    This property specifies the time, in milliseconds, after which the connection times out.

    Default value: 10000

    pg.gateway.elasticsearch.http.socketTimeout

    Mandatory

    This property specifies the wait time, in milliseconds, for a reply once the connection to Elasticsearch is established after which it times out.

    Default value: 30000

    pg.gateway.elasticsearch.http.maxRetryTimeout

    Mandatory

    This property specifies the wait time, in milliseconds, for retries after which it times out.

    Default value: 100000

    It is advisable to set max retry time for a request to (number of nodes * socketTimeOut )+connectionTimeout

    pg.gateway.elasticsearch.http.keepAlive.maxConnections

    Mandatory

    This property specifies the maximum number of persistent connections that can be established between an webMethods API Gateway and Elasticsearch cluster.

    Default value: 50

    pg.gateway.elasticsearch.http.keepAlive.maxConnectionsPerRoute

    Mandatory

    This property specifies the maximum number of persistent connections that can be established per HTTP route to an Elasticsearch server.

    Default value: 15

    pg.gateway.elasticsearch.http.username

    This property specifies the user name to connect to Elasticsearch using basic authentication.

    pg.gateway.elasticsearch.http.password

    This property specifies the password to connect to Elasticsearch using basic authentication.

    pg.gateway.elasticsearch.https.keystore.filepath

    This property specifies the Keystore file path for establishing HTTPS communication with Elasticsearch.

    pg.gateway.elasticsearch.https.truststore.filepath

    This property specifies the truststore file path for establishing HTTPS communication with Elasticsearch.

    pg.gateway.elasticsearch.https.keystore.password

    This property specifies the Keystore password for establishing HTTPS communication with Elasticsearch.

    pg.gateway.elasticsearch.https.keystore.alias

    This property specifies the Keystore alias for establishing HTTPS communication with Elasticsearch.

    pg.gateway.elasticsearch.https.truststore.password

    This property specifies the truststore password for establishing HTTPS communication with Elasticsearch.

    pg.gateway.elasticsearch.https.enabled

    This property specifies whether you want to enable or disable the HTTPS communication with Elasticsearch.

    Default value: false

    If this property is set to false none of the above properties related to HTTPS are respected.

    pg.gateway.elasticsearch.outbound.proxy.enabled

    This property specifies whether you want to enable or disable outbound proxy communication.

    Default value: true

    pg.gateway.elasticsearch.outbound.proxy.alias

    This property specifies the outbound proxy alias name used to connect to Elasticsearch.

    pg.gateway.elasticsearch.https.enforce.hostname.verification

    This property enforces the host name verification for SSL communication.

    Default value: false

    pg.gateway.elasticsearch.sniff.enable

    Mandatory

    This property enables sniffers to add the other nodes in an Elasticsearch cluster to the client so that the client can talk to all nodes.

    Default value: true

    This configuration must be set to false if you are changing the network when webMethods API Gateway or Elasticsearch is running.

    pg.gateway.elasticsearch.tenantId

    This property allows you to specify a tenant name of your choice. This value must be same across all nodes.

    The default value of this property is the Integration Server instance name. So, ensure that you provide same name for all Integration Server nodes in a cluster.

    If you modify this value, you must edit the value in all nodes and restart the webMethods API Gateway server for the change to take effect.

    pg.gateway.elasticsearch.sniff.timeInterval

    Mandatory

    This property enables adding the newly added Elasticsearch cluster nodes to existing REST client in a specified time interval in milliseconds.

    Default value: 60000

  3. Restart API Gateway for the HTTP client to take effect.
    Note: If you start API Gateway before you start Kibana, you must restart API Gateway to see the changes.

You can also externalize the Elasticsearch tenant ID and configuration by using a master configuration file. For details, see Externalizing Configurations.