Upgrade an Installation from the Tarball
Use the same procedure to upgrade an installation from the tarball when Spark runs locally on the Transformer machine or when Spark runs on a cluster.
Step 1. Shut Down the Previous Version
Stop all running pipelines and then shut down the previous version of Transformer.
Step 2. Back Up the Previous Version
Before you install the new version, create a backup of the files in the previous version by copying and renaming the configuration, data, and resource directories. That way, you can continue to run the previous version if needed.
- TRANSFORMER_EXTERNAL_RESOURCES - The Transformer directory for external resources.
- TRANSFORMER_RESOURCES - The Transformer directory for runtime resource files.
-
STREAMSETS_LIBRARIES_EXTRA_DIR - The Transformer directory for external libraries.
For example, if you are upgrading version 3.12.0, back up the Transformer
configuration directory and name it as follows:
/etc/transformer3120
.
Step 3. Install the New Version
Install the new version of the tarball on the same machine as the previous version.
Step 4. Update Environment Variables
Update the Transformer environment configuration file so that the new version of Transformer uses a new configuration directory but the same data, log, resource, Java, and Spark directories as the previous version.
For example, let's say your previous Transformer
version used the directory /var/lib/transformer
to store the data
files for pipeline configuration and run details. When you upgrade, you configure
the new version of Transformer to
use the same working directory /var/lib/transformer
for the data
files. As a result, the new version has access to the pipelines created in the
previous version.
For more information about modifying Transformer environment variables, see Modifying Environment Variables.
Step 5. Update the Configuration Files
A new Transformer version can include new properties and configuration files required for Transformer to start or function properly. In the previous step, you updated the environment
configuration file so that the new version of Transformer uses the new configuration files stored in the $TRANSFORMER_CONF
directory. In this step, you’ll compare the previous and new versions of the
configuration files, and update the new files as needed with the same customized
property values.
For example when upgrading from version 3.12.0, you compare the files in your back up directory, /etc/transformer3120, with the files in the /etc/transformer directory. You update the new files in the /etc/transformer directory with any customizations made in the previous files in the /etc/transformer3120 directory.
Step 6. Start the New Version of Transformer
Start the new version of Transformer, as described in Starting Transformer Manually.
If the previous version of Transformer was registered with Control Hub and you correctly updated the configuration files during the upgrade, then the new version of Transformer is automatically registered and enabled to work with Control Hub.