Upgrading a Highly Available Environment

When you upgrade additional Control Hub instances for a highly available environment, use the shortened upgrade process described here.

1. Install the new version of Control Hub on the same machine as the previous version or on a separate machine.

   Use one of the following installation methods to install the new version of

   Control Hub:
   • Installing from the Tarball for a Manual Start
   • Installing from the Tarball for a Service Start for SysV Init Systems
   • Installing from the Tarball for a Service Start for Systemd Init Systems
   • Installing from the RPM Package
2. If you installed the RPM package on the same machine as the previous version, rename the previous and new versions of the configuration files.

   When you install the new RPM package on the same machine as the previous version, the configuration files are written to the same default directory as the previous version, /etc/dpm. The new versions of the configuration files are renamed with the following extension: .rpmnew. For example, the new version of the Control Hub configuration file is renamed to dpm.properties.rpmnew.

   1. In the working $DPM_CONF directory, /etc/dpm by default, rename all previous configuration files with the following extension: .old.
   2. Remove the following extension from all new configuration files: .rpmnew.
3. Download the JDBC driver for the relational database instance that you are using:
   • MariaDB or MySQL when Control Hub uses Java 8 - Download the MySQL JDBC driver version 5 (5.1.44 or later) from the following location: https://dev.mysql.com/downloads/connector/j/5.1.html
   • MariaDB or MySQL when Control Hub uses Java 11 - Download the MySQL JDBC driver version 8 (8.0.19 or later) from the following location: https://dev.mysql.com/downloads/connector/j/8.0.html
   • PostgreSQL - Download the PostgreSQL JDBC driver version 42.1.4 or later from the following location: https://jdbc.postgresql.org/download/
4. Copy the driver to the following directory:

   $DPM_HOME/extra-lib

   For example, copy the driver to the following directory in an RPM installation:

   /opt/streamsets-dpm/extra-lib

5. Set the DPM_HOME and DPM_CONF environment variables.
   • Use the following command to set the DPM_HOME environment variable:

     export DPM_HOME=<home directory>

     For example:

     export DPM_HOME=/opt/streamsets-dpm 

   • Use the following command to set the DPM_CONF environment variable:

     export DPM_CONF=<configuration directory>

     For example:

     export DPM_CONF=/etc/dpm

6. Copy all files from the $DPM_CONF directory in the initial Control Hub instance that you upgraded to the $DPM_CONF directory in this additional Control Hub instance.
   By copying all configuration files, you ensure that this additional Control Hub instance connects to the same load balancer, databases, SMTP account, and system Data Collector as the initially upgraded Control Hub instance.
7. Update the configuration file for each Control Hub application to define the same unique component ID used by the previous version. Each Control Hub instance of the same version must use a unique component ID.

   We recommend configuring the new version to use the same component ID as the previous version. However, you can configure the new version to use a different component ID. For example, if you install the new version on a different machine and your policy is to set the component ID to the IP address of the machine, you'd want to configure a different component ID for the new version. For example: <application name>199.57.90.24.

   Modify the value of the dpm.componentId property in these files located in the Control Hub configuration directory, $DPM_CONF:

   • connection-app.properties
   • dpm.properties
   • dynamic_preview-app.properties
   • jobrunner-app.properties
   • messaging-app.properties
   • notification-app.properties
   • pipelinestore-app.properties
   • policy-app.properties
   • provisioning-app.properties
   • reporting-app.properties
   • scheduler-app.properties
   • sdp_classification-app.properties
   • security-app.properties
   • sla-app.properties
   • timeseries-app.properties
   • topology-app.properties
8. Modify and then run the security script to generate a unique authentication token for each Control Hub application.
   1. In the command prompt, set the DPM_ADMIN_USER and DPM_ADMIN_PASSWORD environment variables.
      • Use the following command to set the DPM_ADMIN_USER environment variable:

        export DPM_ADMIN_USER=<user name>

        For example:

        export DPM_ADMIN_USER=admin@admin 

      • Use the following command to set the DPM_ADMIN_PASSWORD environment variable:

        export DPM_ADMIN_PASSWORD=<password>

        For example:

        export DPM_ADMIN_PASSWORD=mypassword 

   2. Use a text editor to modify the $DPM_HOME/dev/02-initsecurity.sh script to comment out the last line in the script as follows:

      # create SAML configuration
      # "${DPM_DIST}/bin/streamsets" dpmcli security createSamlConfig -u "${DPM_ADMIN_USER}" -p "${DPM_ADMIN_PASSWORD}"

   3. Save and close the script.
   4. Use the following command to run the security script from the $DPM_HOME directory:

      dev/02-initsecurity.sh <component ID>

      For example, if you defined the component ID for this installation as <application name>002, use the following command:

      dev/02-initsecurity.sh 002

      You do not need to specify the default component ID of 000.

9. If you installed the new version of Control Hub on a separate machine, make sure that the Control Hub instance can access the front end of the load balancer to communicate with the other Control Hub instances.