Upgrading, known issues, and security considerations

Review this section for information about upgrading DR, known issues, and security considerations.

Upgrading SOAR Disaster Recovery

Before you can upgrade the SOAR Platform or DR, you must disable DR and then upgrade the SOAR Platform on each of the appliances. When SOAR Platform and DR are upgraded, the steps include enabling DR, which can take some time and requires downtime on master.

About this task

For the upgrade scenario described here, DR is currently enabled.

Procedure

Disable DR by first stopping the SOAR Platform, waiting until synchronization is complete, and then running the playbook to disable DR, as described in Disabling SOAR Disaster Recovery.
Upgrade the SOAR Platform on each of the appliances, as described in Changed in 51.0.0.0 Upgrading the SOAR Platform.
Upgrade DR using the following command:
```
sudo yum upgrade resilient-dr
```
Verify that the vault_vars_maximum_async_wait_in_minutes and vault_vars_poll_interval_in_seconds are entries in the Ansible® vault files, as described in Step 5: Creating Ansible vault files. If these entries are missing, compare the vault contents with vault.template in /usr/share/resilient-dr/ansible/templates/ and add these and any other entries that have yet to be set in the vault file.

Important: The parameters must be set before running the enable_dr action.
Edit the inventory file and change the ansible_python_interpreter parameter to use the built-in Ansible environment.
```
ansible_python_interpreter: /opt/ansible-venv/python/ansible-python-env-latest/bin/python3
```
Verify that the vault_postgres_base_path and vault_postgres_service_name parameters specify Postgres 14 in the Ansible vault files. If they are set to Postgres 12, compare the vault contents with vault.template in /usr/share/resilient-dr/ansible/templates/ and add these and any other entries that have yet to be set in the vault file.

Important: These parameters must be set correctly before running the enable_dr action.
Enable DR, as described in Enabling SOAR disaster recovery.

Security considerations for Disaster Recovery

It is important to encrypt the SSH key vault file and SSL certs and follow the security practices described in this guide.

Refresh the SSH keys: If you need to refresh the SSH keys, run the enable_dr.yml playbook with new keys in the ssh_vault.yml SSH key vault file. The new keys will be used and the old keys will be removed.
Replace the SSL certs: If you need to replace the SSL certs, manually remove the certs, install the new certs using either the manual or supply method, as described in Step 3: Configuring Postgres for SSL, and run the enable_dr.yml playbook with the new SSL certs in the SSL certs vault files.

Known issues for SOAR Disaster Recovery

This section provides information about any known issues.

Certs not matching

If the root.crt configured on the receiver appliance does not match the server.crt on the master appliance, SSL communication fails. This causes the enable_dr.yml playbook to fail when taking a base backup from the master postgres database and the playbook leaves the receiver database in an archived state. If this happens the playbook attempts to automatically use the created backup archive to restore the database to its original state.

The backup archive file is located in the /crypt directory if you want to manually restore the receiver database. The file name is based on the time stamp, for example, /crypt/database20180702-1635.gz. (database%Y%d-%H%M.gz).

Run actions as resadmin

To avoid changing file ownership, you must run all of the actions, including Ansible-vault decrypt/encrypt, as the resadmin user.

Synchronization problems from master appliance

If there are unexpected problems during synchronization from the master appliance, it can cause the receiver database to be corrupted, and PostgreSQL does not start.

If this problem occurs, complete the following steps to correct it:

Switch to the root user:
```
sudo -i
```
Run the following command to remove the app manager database:
```
rm -rf /crypt/app-manager-database/*
```

Set the environment variable for the database folder path:

PGDATA=$(awk -F= '/^pg1-path/ { print $2}' /etc/pgbackrest/pgbackrest.conf)

Create a new directory in the database path:
```
mkdir $PGDATA
```
Change the owner and group to the Postgres user:
```
chown postgres.postgres $PGDATA
```

Run the following command to restore the database from the previous backup:

sudo -u postgres pgbackrest --stanza=ibm-security-soar --process-max=4 restore