Running system recovery using the service assistant

Start recovery when all node canisters that were members of the system are online and have candidate status. If any nodes display error code 550 or 578, remove system information to place them into candidate status. Do not run the recovery procedure on different node canisters in the same system.

Before you begin

Note: Ensure that the web browser is not blocking pop-up windows. If it does, progress windows cannot open.

Before you begin this procedure, read the recover system procedure introductory information; see Recover system procedure.

About this task

Attention: This service action has serious implications if not completed properly. If at any time an error is encountered not covered by this procedure, stop and call the support center.

Note: Each individual stage of the recovery procedure can take significant time to complete, depending on the specific configuration.

Procedure

Log on to the service assistant.
Select Recover System from the navigation.
Follow the online instructions to complete the recovery procedure.
1. Verify the date and time of the last quorum time. The time stamp must be less than 30 minutes before the failure. The time stamp format is YYYYMMDD hh:mm, where YYYY is the year, MM is the month, DD is the day, hh is the hour, and mm is the minute.
  Attention: If the time stamp is not less than 30 minutes before the failure, call the support center.
2. Verify the date and time of the last backup date. The time stamp must be less than 24 hours before the failure. The time stamp format is YYYYMMDD hh:mm, where YYYY is the year, MM is the month, DD is the day, hh is the hour, and mm is the minute.
  Attention: If the time stamp is not less than 24 hours before the failure, call the support center.
  
  Changes that are made after the time of this backup date might not be restored.

Results

Any one of the following categories of messages might be displayed:

```
T3 successful
```
The volumes are back online. Use the final checks to get your environment operational again.
```
T3 recovery completed with errors
```
T3 recovery completed with errors: One or more of the volumes are offline because there was fast write data in the cache. To bring the volumes online, see Recovering from offline volumes using the CLI for details.
```
T3 failed
```
Call the support center. Do not attempt any further action.

Verify that the environment is operational by completing the checks that are provided in What to check after running the system recovery.

If any errors are logged in the error log after the system recovery procedure completes, use the fix procedures to resolve these errors, especially the errors that are related to offline arrays.

If the recovery completes with offline volumes, run the command-line interface (CLI) svctask recovervdisk command to access the volumes.