Constraints to upgrades
Some constraints exist regarding device upgrades.
- Only one Accesser® device for the same vault can be upgraded at a time.
- During an upgrade, a threshold number of Slicestor®
devices must be available to ensure data availability and to protect data reliability.
The most conservative upgrade approach (and default) is to upgrade a single device per storage pool at a time. For faster upgrades, multiple devices can be upgraded simultaneously by adjusting the Upgrade Throttle parameter from 1 [default] to n. The number of available Slicestor devices equals to [n = Width - (Alert Level + 1)].
If no alert levels are set for any of the vaults in the system, an upgrade throttle of one is used. Intuitively, one corresponds to a "conservative" upgrade, and "n" corresponds to an "aggressive" upgrade. It is possible that the upgrade throttle that is selected cannot be achievable due to the vault configurations across the storage pools. In all circumstances, vault health is never compromised.
- When upgrading devices in a storage pool that uses Concentrated Dispersal vaults, the upgrade process waits 72 hours between
initiating each device upgrade.
This is done to ensure data integrity. Once the 72-hour window has elapsed, another device can begin upgrade. This will also apply to storage pools operating in a mirrored setup where if a device on one side of the mirror has upgraded, all other devices on both sides of the mirror must wait 72 hours.
In the
configuration options, the Automatic Shutdown Timeout parameter can be used
to specify a maximum time devices wait for outstanding transactions to complete when attempting to
cleanly shut down. This parameter is not specific to the upgrade process and applies to all stops of
the dsnet-core service but it is of particular importance in the context of system
upgrades.
By default, this parameter is set to None, which means that devices wait
indefinitely for transactions to close before shutting down cleanly. It prevents the possibility of
inducing write errors as a result of the upgrade process and can lead to unbounded upgrade times if
long-standing I/O operations are being performed on the system. The Automatic Shutdown
Timeout can be used to place an upper bound on the amount of time the
dsnet-core process waits for the transaction to close before forcefully ends
outstanding transactions and completing the shutdown process.
The following options are available:
- None
- 15 min
- 30 min
- 1 hour
- 2 hours
- 4 hours
- 8 hours
- 16 hours
Selecting a value other than None prompts a warning message that indicates
active writes are dropped and in-flight data that is not yet committed to the system might be
lost.
When a device begins the graceful shutdown of the dsnet-core process in
preparation for upgrade, it waits up to the selected timeout for the shutdown to complete. If
shutdown completes before the timeout, the device upgrade proceeds normally. If not, after timeout
expiration, automatic termination will be performed on the dsnet-core process, and
any active writes will be canceled.
On the right side of the Manager Web Interface under the upgrade
queue, for the device being upgraded, a countdown indicates the amount of time that is left before
automatic termination. This operation can be changed or stopped by setting the Automatic Shutdown
Timeout to a different value or None. If a new value is specified (something other
than None), any previous time that is spent in the stopping state is accounted
for.