Upgrading API Gateway cluster

Edit online

About this task

Note: This procedure is applicable only for on-premise installation.
Prerequisites:
  • Install latest fixes in both source and target versions.
    Note: The version of target API Gateway must be higher than the source API Gateway instance. Source API Gateway versions of 10.3 and later can be upgraded to API Gateway 10.15.
  • If custom keystore or truststore files are used in the source API Gateway installation, copy the files to the same location in the target installation.
Note: API Gateway does not support migrating from one operating system to another (for example, from a Windows system to a UNIX system, or the other way around).

The following image illustrates the different stages in upgrading an API Gateway cluster:

The API Gateway migration utility provides the support to:

  • Separately migrate Elasticsearch and API Gateway file configurations.
  • Migrate data from externally configured Elasticsearch.
  • Log all the migration data in a single file, that is migrationLog.txt.
  • Revert in case of failure in Elasticsearch data migration.
Note: In the API Gateway versions 10.2 and above, the folder name EventDataStore is changed to InternalDataStore. Throughout this section, the source API Gateway installation directory is referred as <SOURCE> and the target API Gateway installation directory is referred as <TARGET>.

To upgrade an API Gateway cluster

Procedure

  1. Configure the path.repo property in the source elasticsearch.yml file located at <SOURCE>\InternalDataStore\config. Make sure that the path.repo is a shared network folder and is accessible for all the Elasticsearch nodes in the cluster.
  2. Configure the reindex.remote.whitelist property in the target elasticsearch.yml file for re-indexing the data in the target Elasticsearch. This YAML file is located at <TARGET>\InternalDataStore\config. The reindex.remote.whitelist property copies the documents from the <SOURCE> to <TARGET> Elasticsearch instances.

    Syntax for reindex.remote.whitelist property in the target elasticsearch.yml file is as follows: 

    reindex.remote.whitelist: <source_host>:<source_port>
    Note:
    • The reindex.remote.whitelist value in the target elasticsearch.yml file must match the value of the pg.gateway.elasticsearch.hosts property present in the config.properties file located at <SOURCE>\IntegrationServer\instances\default\packages\WmAPIGateway\config\resource\elasticsearch.
    • For managed Elasticsearch instances, you do not have to configure the reindex.remote.whitelist property.

    An example for the reindex.remote.whitelist property in the elasticsearch.yml file is as follows: 

    
cluster.name: SAG_EventDataStore
node.name: SAG-1YVHZY21633236119876
path.logs: C:\SoftwareAG_1011\InternalDataStore/logs
network.host: 0.0.0.0
http.port: 9240
discovery.seed_hosts: ["hostname1:9340","hostname2:9340","hostname3:9340"]
transport.tcp.port: 9340
path.repo: ['C:\SoftwareAG_1011\InternalDataStore/archives']
cluster.initial_master_nodes: ["node1","node2","node3"]
reindex.remote.whitelist: localhost:9240
  3. Optional. Configure the source Elasticsearch host and port details in the config.properties file located at <SOURCE>\IntegrationServer\instances\default\packages\WmAPIGateway\config\resources\elasticsearch.

    Syntax for Elasticsearch host and port details in the source config.properties file is as follows: 

    pg.gateway.elasticsearch.hosts=<source_host>:<source_port>

    Configure the source Elasticsearch host and port details for all the API Gateway nodes in the cluster.

    Note: This step is applicable only if the source API Gateway version is 10.1. From versions 10.2 and above, the Elasticsearch host and port values are populated automatically.
  4. Configure the basic authentication credentials to connect to the source Elasticsearch, if the source Elasticsearch is protected.

    To configure the basic authentication credentials, add the following properties to the config.properties file located at <SOURCE>\IntegrationServer\instances\default\packages\WmAPIGateway\config\resources\elasticsearch: 

    
pg.gateway.elasticsearch.http.username=<username>
pg.gateway.elasticsearch.http.password=<password>

    Configure the basic authentication credentials for all the API Gateway nodes in the cluster.

  5. Configure the certificates to connect to the source Elasticsearch.

    If the source Elasticsearch is protected with HTTPS, add the source certificates (public key) into the target Elasticsearch JVM's truststore. For example, in case of API Data Store, add the public keys to the truststore cacerts file located at <TARGET>\jvm\jvm\jre\lib\security. Do for all the target API Gateway nodes.

    Note: If, external Elasticsearch is used for the target API Gateway, then the certificates must be imported to its corresponding JVM.

    Sample command to import the truststore of the source Elasticsearch into the target Elasticsearch JVM is as follows: 

    <TARGET>\jvm\jvm\bin\keytool -import -keystore <TARGET>\jvm\jvm\jre\lib\security\cacerts -file <truststore.jks> -alias <alias>
    Property Description
    truststore.jks Truststore used in <SOURCE> Elasticsearch. Provide the full path of the truststore.

    For 10.1, it is be available at <SOURCE>\WmAPIGateway\config\resources\bean\gateway-es-store.xml and the property is <prop key="searchguard.ssl.http.truststore_filepath"\>sagconfig/root-ca.jks\</prop>

    For 10.2 and above, it is available at <SOURCE>\WmAPIGateway\config\resources\elasticsearch\config.properties and the property is pg.gateway.elasticsearch.https.truststore.filepath.

    Example: sagconfig/root-ca.jks

    alias Alias used in <SOURCE> Elasticsearch.

    Example: wm.sag.com

  6. Start both the source and target Elasticsearch instances and make sure that API Gateway (Integration Server) instances are not started.
    Note: Avoid port conflict. If the source and target API Gateway instances are running in the same machine, then you might not be able to start both the source and target Elasticsearch instances in parallel with the default port configurations. In this case, the target Elasticsearch instance port can be changed temporarily for performing the migration. Both, HTTP and TCP ports must be changed. To change the target Elasticsearch instance ports:
    1. Open the elasticsearch.yml file located at <TARGET>/InternalDataStore/config. Change the value of the HTTP port in the http.port property, and TCP port in the transport.tcp.port property.
    2. Open the config.properties file located at <TARGET>/IntegrationServer/instances/default/packages/WmAPIGateway/config/resources/elasticsearch and find the pg.gateway.elasticsearch.hosts property. If the property is set to changeOnInstall, then do not make further changes to property. If a port is configured, then update it to a new value.

    Revert the temporary target Elasticsearch instance ports to the default port configurations, after completing the migration and the target Elasticsearch instance is shutdown.

    If you are reverting the elasticsearch ports in the elasticsearch.yml file after the migration, then you need to specify the same ports in the config.properties file located at <TARGET>/IntegrationServer/instances/default/packages/WmAPIGateway/config/resources/elasticsearch.

  7. Migrate the API Gateway assets and the analytics data, based on volume of data you are migrating. Go to <TARGET>\IntegrationServer\instances\default\packages\WmAPIGateway\bin\migrate and run any one of the following migration utility command.
    Note: Before running the migration utility command, set up the target Elasticsearch cluster and make sure that all the source and target Elasticsearch instances in the clusters are up and running. Also, do not start the API Gateway instances on any of the nodes.
    1. If the API Gateway analytics data volume is less, migrate the whole Elasticsearch or API Data Store data. The migrate.bat datastore command to migrate the whole Elasticsearch or API Data Store data is as follows: 
      migrate.bat datastore -dstoreSrc <full path to source Elasticsearch config.properties>
      Note: You should run the migrate.bat datastore command in only one target API Gateway node of the cluster.
      Property Description
      dstoreSrc

      If the source and target API Gateway instances are running on the same machine, provide the location where <SOURCE> config.properties file is located. Sample command is as follows: 

      migrate.bat datastore -dstoreSrc<SOURCE>\IntegrationServer\instances\default\packages​\​
WmAPIGateway\config\resources\elasticsearch\config.properties

      If the source and target instances are running on different machines, then the source installation directory or the Elasticsearch config.properties file must be shared in the network. Otherwise, copy and paste the source config.properties file to the shared location. Sample command is as follows: 

      migrate.bat datastore -dstoreSrc\chebackup01\installations\source\IntegrationServer\instances\default\
packages\WmAPIGateway\​config\resources\elasticsearch\config.properties
    2. If the API Gateway analytics data volume is high, migrate only the core data using the migrate.bat reindex command, and migrate the analytics data using the backup and restore command.
      Note: This step is applicable only, when the source API Gateway version is greater than or equal to 10.11, and is not applicable when the source API Gateway version is less than or equal to 10.7.

      Migrate only the core data

      The migration utility command to migrate only the core data is as follows: 

      migrate.bat reindex 
-sourceESHostUrl <source Elastic search host url>
      Property Description
      sourceESHostUrl The source Elasticsearch or API Data Store URL. Sample command is as follows: 
      migrate.bat reindex -sourceESHostUrl localhost:9240

      Migrate the analytics data using backup and restore

      The steps to migrate the analytics data using backup and restore command is as follows:

      • Take a backup of the analytics data from the source API Gateway instance using the following command: 
        <SOURCE>\IntegrationServer\instances\​​default\packages\WmAPIGateway\​cli\bin\apigatewayUtil
create backup ​​-include ​analytics ​-name <backup_file_name>
        For more information on taking backup, see Creating API Data Store Backup.
      • Copy the backup file to the target API Gateway instance path.repo folder configured under <TARGET>\IntegrationServer\InternalDataStore\config\elasticsearch.yml.
      • Restore the backup data in the target API Gateway instance using the following command: 
        <TARGET>\IntegrationServer\instances\​​default\packages\WmAPIGateway\​cli\bin\apigatewayUtil 
restore backup ​​-include ​analytics ​-name <backup_file_name>.
        For more information on restoring backup, see Restoring Data Store Backup.
  8. Migrate the platform configurations.

    To migrate the platform configurations using externalized configurations, you can use the migrate.bat apigateway command to migrate the platform configurations.

    The migrate.bat apigateway command facilitates the migration of Integration Server-specific settings and configurations. Additionally, it performs data transformation due to variations in the structure or model of data across different versions of API Gateway. These differences may include merged fields or the addition of new ones in specific versions. As a result, the migrate.bat apigateway command is essential to trigger the necessary data transformation after a datastore migration for the application to function as intended.

    To use the migrate.bat apigateway command, go to <TARGET>\IntegrationServer\instances\default\packages\WmAPIGateway\bin\migrate and run the following command. 

    migrate.bat apigateway -srcDir <SOURCE> -instanceName <source instance name> -newInstanceName <target instance name>
    Note: You should run the migrate.bat apigateway command in all the target API Gateway nodes of the cluster.
    Property Description
    srcDir

    If the source API Gateway instance is installed on the same machine, provide the source API Gateway installation directory. Sample command is as follows: 

    -srcDir C:\installations\source

    If the source API Gateway instance is installed on a different machine, then share the entire installation folder. Sample command is as follows: 

    -srcDir \chebackup01\source
    instanceName Optional.

    Provide the <SOURCE> instance name that you want to migrate. If you do not provide any name, then the default instance is used. Sample command is as follows: 

    -instanceName test
    newInstanceName Optional.

    Provide the <TARGET> instance name that you want to migrate to. If you do not provide any name, then the default instance is used. If you have created a new instance other than the default in Integration Server and you want to migrate to the new instance then provide its name. Sample command is as follows: 

    -newInstanceName prod
  9. Perform these steps after migrating the platform data and platform logs.
    1. Shutdown the target Elasticsearch instance.
    2. Start the target API Gateway server.

    The API Gateway upgrade is complete.

    • You can find the logs at the target directory <TARGET>/install/logs/migrationLog.txt.
    • You can find the API Gateway migration reports at <TARGET>\install\logs\APIGW_Migration_Reports_<date_time>.