Upgrading external Elasticsearch in API Gateway

When considering an upgrade to Elasticsearch, it is necessary to ensure compatibility between the webMethods API Gateway and both the current and target versions of external Elasticsearch. This section explains how to perform external Elasticsearch upgrades to ensure both backward and forward compatibility.

When upgrading from one external Elasticsearch version to the next, it is necessary to transfer the core and transcational data from the previous version to the target version. Perform the following steps to copy external Elasticsearch data to the target destination:

1. Open your REST client.
2. Provide the details of the external Elasticsearch endpoints for both the core data and the analytics data in the Body section.

   Configure the following parameters in the REST client by providing the required values. These parameters define the target endpoint where the external Elasticsearch data should be copied.

   • targetCoreESEndpointConfig: Specifies the configuration for the target core external Elasticsearch endpoint. By specifying the values for this parameter, only the core external Elasticsearch data is copied to the target endpoint. To configure this parameter, you can use the following settings by providing the required values.

     Settings	Description	Example
     endpoint	Specifies the URL of the target core external Elasticsearch endpoint.	"endpoint":"http://targetendpoint:9200"
     username	Specifies the username for authentication.	"userName":"abc"
     password	Specifies the password for authentication.	"password":"xyz"

   • targetEventsAndLogsESEndpointConfig: Specifies the configuration for the target events and logs external Elasticsearch endpoint. By specifying the values for this parameter, only the transactional external Elasticsearch data is copied to the target endpoint. To configure this parameter, you can use the following settings by providing the required values.

     Settings	Description	Example
     endpoint	Specifies the URL of the target events and logs external Elasticsearch endpoint.	"endpoint":"http://targetendpoint:9200"
     username	Specifies the username for authentication.	"userName":"xyz"
     password	Specifies the password for authentication.	"password":"abc"

   • targetDashboardConfig: Specifies the configuration for the target Dashboard endpoint external Elasticsearch endpoint. By specifying the values for this parameter, the endpoint is directed to the target external Kibana dashboard, which has the analytics data.To configure this parameter, you can use the following settings by providing the required values.

     Settings	Description	Example
     endpoint	Specifies the URL of the target Dashboard endpoint.	"endpoint":"http://targetendpoint:5601"

     {
         "targetCoreESEndpointConfig": {
             "endpoint": "http://targetendpoint:9200",
             "userName": "xyz",
             "password": "abz",
         },
         "targetEventsAndLogsESEndpointConfig": {
             "endpoint": "http://targetenpoint:9200",
             "userName": "xyz",
             "password": "abc",
         },
         "targetDashboardConfig": {
             "endpoint":"http://targetendpoint:5601"
         }
     }

   • Copy the core data individually by configuring and providing the values for the 'targetCoreESEndpointConfig' parameter in the REST client and making a POST call to the follwing URL:

     POST/rest/apigateway/migrate/datacopy/core

     Similarly, copy the events and logs data individually by configuring and providing the values for the 'targetEventsAndLogsESEndpointConfig' and 'targetDashboardConfig' parameters in the REST client and making a POST call to following URL:

     POST/rest/apigateway/migrate/datacopy/eventsandlogs

1. To communicate with a secured external Elasticsearch using Keystore and Truststore, configure the following parameters in the REST client by providing the required values:

   • keystoreConfig: Specifies the configuration for the target Keystore endpoint. By specifying the values for this parameter, the target external Elasticsearch server validates the client certificate and key stored in the Keystore. To configure this parameter, you can use the following settings by providing the required values.

     Settings	Description	Example
     filepath	Specifies the file path of the target core/events and logs Keystore endpoint.	"filePath": "c:/keystorefiles/coreelasticks.jks" or "filePath": "c:/keystorefiles/eventelastic.jks"
     alias	Specifies the alias for authentication.	"alias":"abc"
     password	Specifies the password for authentication.	"password":"xyz"

   • truststoreConfig: Specifies the configuration for the target Truststore endpoint. The Truststore validates the target external Elasticsearch server certificate. To configure this parameter, you can use the following settings by providing the required values.

     Settings	Description	Example
     filepath	Specifies the file path of the target core/events and logs Truststore endpoint.	"filePath": "c:/truststorefiles/coreelasticts.jks" or "filePath": "c:/truststorefiles/eventelasticts.jks"
     password	Specifies the password for authentication.	"password":"xyz"

     For example,

     {
         "targetCoreESEndpointConfig": {
             "endpoint": "https://targetendpoint:9200",
             "userName": "xyz",
             "password": "abz",
             "keystoreConfig": {
                 "filePath": "c:/keystorefiles/coreelasticks.jks",
                 "alias": "hello",
                 "password": "abcd"
             },
             "truststoreConfig": {
                 "filePath": "c:/truststorefiles/coreelasticts.jks",
                 "password": "abcd"
             }
         },
         "targetEventsAndLogsESEndpointConfig": {
             "endpoint": "https://targetendpoint:9200",
             "userName": "xyz",
             "password": "abc",
             "keystoreConfig": {
                 "filePath": "c:/keystorefiles/eventelastic.jks",
                 "alias": "helloa",
                 "password": "abcc"
             },
             "truststoreConfig": {
                 "filePath": "c:/truststorefiles/eventelasticts.jks",
                 "password": "abcc"
             }
         },
         "targetDashboardConfig": {
             "endpoint":"https://targetendpoint:5601"
         }
     }

2. Make a call to the following URL, with the IP address of the system where webMethods API Gateway is installed and press Enter:

   POST/rest/apigateway/migrate/datacopy

   The data copy process will start.

Throughout the upgrade process, you can retrieve the status, which can be valuable for troubleshooting any potential issues.

Upon successful data copying, the configuration files is automatically reflect the changes by pointing to the new cluster. You can verify the same in the following files:

• SAGInstaller/msr/Integration Server/packages/WmAPIGateway/config/resources/elasticsearch/config.properties
• SAGInstaller/msr/Integration Server/packages/WmAPIGateway/config/resources/elasticsearch/analyticsds.properties

In the event of an unsuccessful upgrade, a rollback option is available. The configuration changes is reverted, and the client is directed back to the source external Elasticsearch cluster. Perform the following steps to initiate the rollback process:

1. Open your REST client.
2. Provide information regarding source end points for each external Elasticsearch in the Request section.

   Configure the following parameters in the REST client by providing the required values. These parameters define the target endpoint where the external Elasticsearch data should be copied.

   • targetCoreESEndpointConfig: Specifies the configuration for the source core external Elasticsearch endpoint. By specifying the values for this parameter, only the core external Elasticsearch data is copied to the source endpoint. To configure this parameter, you can use the following settings by providing the required values.

     Settings	Description	Example
     endpoint	Specifies the URL of the source core external Elasticsearch endpoint.	"endpoint":"http://sourceendpoint:9200"
     username	Specifies the username for authentication.	"userName":"abc"
     password	Specifies the password for authentication.	"password":"xyz"

   • targetEventsAndLogsESEndpointConfig: Specifies the configuration for the source events and logs external Elasticsearch endpoint. By specifying the values for this parameter, only the transactional external Elasticsearch data is copied to the source endpoint. To configure this parameter, you can use the following settings by providing the required values.

     Settings	Description	Example
     endpoint	Specifies the URL of the source events and logs external Elasticsearch endpoint.	"endpoint":"http://sourceendpoint:9200"
     username	Specifies the username for authentication.	"userName":"xyz"
     password	Specifies the password for authentication.	"password":"abc"

   • targetDashboardConfig: Specifies the configuration for the source Dashboard endpoint external Elasticsearch endpoint. the endpoint is directed to the source external Kibana dashboard, which has the analytics data.To configure this parameter, you can use the following settings by providing the required values.

     Settings	Description	Example
     endpoint	Specifies the URL of the source Dashboard endpoint.	"endpoint":"http://sourceendpoint:5601"

     For example,

     {
         "targetCoreESEndpointConfig": {
             "endpoint": "http://sourceendpoint:9200",
             "userName": "xyz",
             "password": "abz",
         },
         "targetEventsAndLogsESEndpointConfig": {
             "endpoint": "http://sourceenpoint:9200",
             "userName": "xyz",
             "password": "abc",
         },
         "targetDashboardConfig": {
             "endpoint":"http://sourceendpoint:5601"
         }
     }

   You can provide the source Keystore/Truststore values to get the source data.

3. Make a call to the following URL, with the IP address of the system where webMethods API Gateway is installed and press Enter:

   POST/rest/apigateway/migrate/datacopy/rollback