Database failover capabilities vary during a cluster upgrade

Symptom

Database	Behavior
Distributed session cache	If the primary master fails, failover goes to secondary master. Changes are done in the secondary master and reconciliation occurs when primary master is restored.
Runtime database
Configuration database	If the primary master fails, there is no failover to the secondary master. No changes are possible on the primary or secondary master until the primary master is back online.

Issue 1:
When a high availability cluster is active, a situation exists during the firmware upgrade on the primary node where the configuration database is read-only.

A read-only database prevents the upgrade process from writing to the configuration database when creating new tables, modifying schema of existing tables, and inserting or updating rows on tables.

The reason the database on the primary node becomes read-only is the firmware upgrade requires the appliance to be rebooted. During a reboot, the high availability controller switches the secondary master to be read-write and act as the temporary primary master.

When the primary node reboots and the database starts:

1. It recognizes that the secondary node is in control and starts in read-only mode.
2. The appliance cluster manager includes a background thread which will eventually switch the primary node to resume its role as the primary master database.
3. The database on the primary node becomes writeable.

However, during an upgrade, the database upgrade scripts are executed before the primary database has become writeable.

Issue 2:
For the distributed session cache and runtime database, a situation exists where changes to the secondary databases are not reflected in the primary database after the completion of a cluster upgrade.