Upgrading a Faspex 4.X server with a remote database

Upgrade from a version before Faspex 5.0 and confirm the persistence of files and data.

Note: IBM Aspera Faspex 4.x will no longer be supported after September, 2023.

Warning: Do not continue with this upgrade until you have reviewed all the breaking changes and upgrade considerations.

Important:

Before upgrading to Faspex 5 you must be on Faspex 4.4.2 PL4. Refer to the IBM Aspera Faspex 4.4 Admin Guide for instructions on how to upgrade Faspex 4.

Review breaking changes and upgrade considerations

Attention: Review the Secret key management and password protection section before upgrading Faspex 5.

Operating system

Faspex 5 does not support Windows as an operating system. If you're currently using Windows, in order to upgrade to Faspex 5, migrate your Faspex 4.X instance to a Linux server and test before upgrading to Faspex 5 on Linux.

Your server must run CentOS7, RHEL 7, RHEL 8, or (starting in 5.0.4) Amazon Linux 2.

Your server must have Docker or (starting in 5.0.4) Podman 4.1 that is installed. For tips on installing Docker and Podman, see Tips for installing Docker or Podman.

Note: For Podman and operating system compatibility, see the Faspex 5 Release notes.

Your server must also have network access to the IBM Cloud Container Registry (icr.io). The installer pulls container images from icr.io.

Database

If you are using the default Faspex 4 database, the upgrade process migrates your data to the faspex-db container. If you have an external database running, you will have to back up your Faspex 4 external database and run migration against the external database.

Note: The following error is displayed during the upgrade: "Column count of mysql.proc is wrong. Expected 21, found 20. Created with MariaDB 50737, now running 100803. Please use mariadb-upgrade to fix this error" This is expected to happen and the installer will continue without any issues.

Directory services

For security reasons, Faspex 5 does not support directory services. You must instead front your directory service with a SAML Identity Provider (IdP) and use SAML based authentication for your users. Do not upgrade if you rely on a direct connection between Faspex and directory services. If you have Directory Services users, you will need to migrate those users to SAML before upgrading from Faspex 4 to Faspex 5.

High-Speed Transfer Server version requirement

Faspex 5 uses a more modern API for interacting with IBM Aspera High-Speed Transfer Server (HSTS). Faspex 5 uses the activity logging feature available on HSTS 4.3 or later. You must upgrade HSTS on your nodes and enable activity logging before upgrading (see Enable activity logging on a HSTS node).

For setups where collocation of Faspex 5 with HSTS is unavoidable, the IP address included in the node configuration in Faspex 5 can no longer be 127.0.0.1 or localhost. This is because Faspex 5 is containerized and localhost refers to the container itself, not the server that is running the container. Use the private IP address or FQDN of the server instead.

Connect version

Faspex 5 requires users run Connect version 4.1.3 or later and does not currently support locally hosting the Connect SDK.

Do not upgrade if your users rely on downloading Connect from the Faspex server (instead of the Cloudfront CDN). Hosting the Connect SDK and installers locally is expected in a future version of Faspex 5.

If your users cannot upgrade to Connect 4.1.3 and later, you can default users to transfer with HTTP Gateway.

Email addresses

Faspex 5 requires new users to have a unique email address. To log in to new Faspex 5 accounts, you must log in using the account email address. You can still log in using usernames for accounts created before the upgrade.

On upgrade, Faspex automatically reconciles external users that have the same email address as a regular user or a SAML user. While new Faspex users must log in using their email address, users created before the upgrade can still log in to those accounts with their usernames (but not with their email addresses).

After upgrading, you can see a list of users with duplicate email addresses by logging in to the Faspex Utility application. Then, from the Admin application, you can manually decide to keep, edit, or remove accounts with duplicate email addresses (see Review accounts with duplicate email addresses).

New users must change their password on first login

New Faspex 5 users must change their password on first login. You can no longer disable this requirement in server settings.

SAML

The SAML metadata and callback URL routes are different from previous versions. After upgrading, retrieve the new metadata and callback URLs from Faspex and update your SAML Identity Provider (see Reconfigure SAML after upgrading to Faspex 5).

Customization

Faspex 5 no longer supports custom CSS, custom HTML or custom Ruby. Use the branding options available in Configurations > Display settings (see Configure display settings).

Faspex 5 no longer supports the faspex.yml configuration file. If you rely on options in faspex.yml that cannot be set in another way in Faspex 5, then do not upgrade. For a list of supported and unsupported options, see faspex.yml options in Faspex 5.

Faspex 5 does not currently support out-of-transfer file validation or post-processing scripts. Starting in 5.0.4, Faspex supports package processing webhooks to notify external systems (through a POST request) that a package is ready for download.

Integration

Faspex 5 no longer provides rake tasks for automating tasks. Instead, use the Faspex 5 API to perform automation.

The Faspex 5 API is not backwards-compatible with prior versions of the APIs.

Public submission links

Public submission links and external user invitations created in Faspex 4 do not work in Faspex 5. Admins can resend the invitation as long as the invitation is not expired

Note: For a full list of differences, see Differences and breaking changes between Faspex 5 and previous versions.

If you have questions or issues with the upgrading process from Faspex 4 to Faspex 5, after reviewing the breaking changes and upgrade considerations, contact IBM support.

Perform the following steps if you have already set up a blank Faspex 5 environment before upgrading from Faspex 4:

Drop the Faspex table before properly upgrading from Faspex 4.

Revert the service.env file to its original state:

cp /opt/aspera/faspex/conf/docker/original/service.env /opt/aspera/faspex/conf/docker

Upgrade steps

Warning:

Follow these steps before performing any upgrade:

Perform a full environment back up and ensure the back up is successful. In case the upgrade fails, the only reliable, short-term fix is to roll back the environment using the back up.
Test the upgrade in a test environment comparable to the production environment.
If upgrading the test environment is successful, upgrade the production environment, but do not bring the production environment back online.
Prior to bringing the production environment back online, the customer must test the application to determine if an immediate rollback is needed. Otherwise, customers risk losing all data generated between upgrade and rollback.

Note: Refer to the Enabling TLS to connect to the database section if you need to connect to the database using TLS.

Back up your Faspex 4.X database.
```
asctl faspex:backup_database
```
Backup your /opt/aspera/faspex folder entirely.
For each HSTS node, upgrade the IBM Aspera High-Speed Transfer Server version to 4.3 and enable activity logging. You must do this before upgrading Faspex to retain package information. For instructions, see Enable activity logging on a HSTS node.

If your Faspex 4 server is collocated with a HSTS node, you will need to add it again post-upgrade.

If you are setting the token encryption key for users in your aspera.conf file, use dynamically generated token encryption keys instead. For more information, see the IBM Aspera High-Speed Transfer Server:Secrets Management with askmscli section.
If you have a co-located HSTS node

Note: You should change the node host pre-upgrade, but it's possible to do this after upgrading as well.
Install these Podman components: podman-docker, podman-plugins, and skopeo.
```
yum install podman-docker podman-plugins skopeo
```
Download the newest installer and install the RPM file:
```
rpm -Uvh ibm-aspera-faspex-version.build.x86_64.rpm
```
Configure the generated database environment variables configured by the .env files in /opt/aspera/faspex/conf/docker. Refer to the Installing Faspex with a remote database section and select your version of Faspex 5 to follow the correct steps.
Run faspexctl setup.

Agree to upgrade from Faspex 4.X and agree to clean the filesystem. Make sure you respond no (n) when the installer prompts you to install the db container.
Confirm all containers are present and running:
```
faspexctl status
```
Confirm the presence of legacy files and old database files:
Legacy files are stored at /opt/aspera/faspex_legacy.
Old database files are stored at /opt/aspera/faspex/backup/.
Restore your Faspex 4 database backup using the Faspex Utility application:
1. Log in to Faspex Utility.
2. Go to Manage database.
```
faspexctl start utility
```
3. In the Backup or restore the database area, select Restore and select the database file and click Restore.
Migrate the database:
```
faspexctl setup
```
Confirm all containers are present and running:
```
faspexctl status
```
Confirm the upgrade persisted the data from your database. Run:
```
faspexctl exec utility "mysql faspex -u\$FASPEX_DB_USERNAME -p\$FASPEX_DB_PASSWORD -e 'show tables;'"
```
You should be able to access all data from before the upgrade.
In order to migrate your SSL certificates installed, you must copy them from the faspex_legacy folder to the Nginx configuration folder:

Note: If you are using the self-signed certificate from Faspex 4, you should NOT replace the certificates that Faspex 5 created.
1. Rename and move your certificate file. Replace server_cert_file with either server-ca.crt or server.crt as needed:
```
cp -iv /opt/aspera/faspex_legacy/conf/server_cert_file /opt/aspera/faspex/conf/nginx/cert.pem
```
2. Rename and move your key file (server.key):
```
cp -iv /opt/aspera/faspex_legacy/conf/server.key /opt/aspera/faspex/conf/nginx/key.pem
```
3. Restart faspex-router:
```
faspexctl restart router
```
Update your minimum version of Connect to 4.1.3 or later. Log in to the Faspex UI, go to Transfer options and update Minimum Connect version.