Restoring volume snapshots

If you are using Portworx storage and you backed up your Cloud Pak for Data persistent volumes with the Cloud Pak for Data volume backup and restore utility by creating volume snapshots, use this information to restore the snapshots.

About this task

If a service requires additional steps to be performed before a volume snapshot is restored, you can manually scale down all application Kubernetes resources by using the quiesce command. After you perform the additional steps, you can manually scale up the resources after you restore the snapshot by using the unquiesce command. For the quiesce command, specify the --force option so all resources are scaled down before you restore a snapshot.

You can restore snapshots in two ways:

Manually scale down resources, restore the snapshot, and then manually scale up resources.
Automatically scale down resources, restore the snapshot, and automatically scale up resources with a single command.

Best practice: You can run the commands in this task exactly as written if you set up environment variables. For instructions, see Setting up installation environment variables.

Ensure that you source the environment variables before you run the commands in this task.

For more information about the Cloud Pak for Data volume backup and restore utility, including a list of commands that you can run, see the cpd-cli backup-restore reference documentation.

Procedure

If you are using one of the following services, do the appropriate pre-restore steps.

Service Preprocessing steps

RStudio® Server Runtimes Delete Kubernetes objects that were created for users.

Service	Preprocessing steps
RStudio® Server Runtimes	Delete Kubernetes objects that were created for users.
SPSS® Modeler	Delete Kubernetes objects by running the following script: `set -xa kubectl delete pod --selector app.kubernetes.io/name=flow-session kubectl delete secret --selector app.kubernetes.io/name=cache-db kubectl delete secret --selector app.kubernetes.io/name=flow-session kubectl delete pod --selector app=flow-session kubectl delete secret --selector app=cache-db kubectl delete secret --selector app=flow-session kubectl delete svc -l type=spss-modeler kubectl delete secrets -l component=spss-modeler kubectl delete deployment -l component=spss-modeler`
Watson™ Studio	Restoring Watson Studio.

SPSS® Modeler

Delete Kubernetes objects by running the following script:

set -xa
kubectl delete pod --selector app.kubernetes.io/name=flow-session
kubectl delete secret --selector app.kubernetes.io/name=cache-db
kubectl delete secret --selector app.kubernetes.io/name=flow-session
kubectl delete pod --selector app=flow-session
kubectl delete secret --selector app=cache-db
kubectl delete secret --selector app=flow-session
kubectl delete svc -l type=spss-modeler
kubectl delete secrets -l component=spss-modeler
kubectl delete deployment -l component=spss-modeler

Watson™ Studio Restoring Watson Studio.

To restore a snapshot by manually scaling down resources, restoring the snapshot, and manually scaling up resources, do the following steps.
1. Manually scale down application Kubernetes resources:
```
cpd-cli backup-restore quiesce -n ${PROJECT_CPD_INSTANCE} --force
```
2. Check for completed jobs and pods by running the snapshot restore command with the --dry-run option, specifying a restore name identifier.
  The --dry-run option reports jobs or pods that are still attached to the PVCs to be restored.
  
  Note: The restore name identifier must consist of lowercase alphanumeric characters or the hyphen (-), and must start and end with an alphanumeric character. The underscore character (_) is not supported.
```
cpd-cli backup-restore snapshot-restore create <restore_name> -n ${PROJECT_CPD_INSTANCE} --from-snapshot <snapshot_name> --dry-run
```
3. If the dry run reports completed or failed jobs, or pods, that reference PVCs, delete them.
  
  Tip: Consider saving the job/pod yaml before you manually delete them, or include the --cleanup-completed-resources option in the restore step.
4. Run the restore command with the --skip-quiesce option:
```
cpd-cli backup-restore snapshot-restore create <restore_name> -n ${PROJECT_CPD_INSTANCE} --from-snapshot <snapshot_name> --skip-quiesce=true
```
5. Manually scale up application Kubernetes resources:
```
cpd-cli backup-restore unquiesce -n ${PROJECT_CPD_INSTANCE}
```
To automatically scale down resources, restore a snapshot, and automatically scale up resources, do the following steps.
1. Run the following snapshot restore command, specifying a restore name identifier.
  Note: The restore name identifier must consist of lowercase alphanumeric characters or the hyphen (-), and must start and end with an alphanumeric character. The underscore character (_) is not supported.
```
cpd-cli backup-restore snapshot-restore create <restore_name> -n ${PROJECT_CPD_INSTANCE} --from-snapshot <snapshot_name>
```
2. If the restore fails because there are completed or failed jobs, or pods, that reference PVCs, delete them, and rerun the restore command.
  
  Tip: Consider saving the job/pod yaml before you manually delete them, or include the --cleanup-completed-resources option in the restore command.
3. If the restore does not automatically scale up resources because of a previous failure, manually scale up resources:
```
cpd-cli backup-restore unquiesce -n ${PROJECT_CPD_INSTANCE}
```

To check the status of a restore, run the following command:

cpd-cli backup-restore snapshot-restore status <restore_name> -n ${PROJECT_CPD_INSTANCE}

To view a list of existing restores, run the following command:

cpd-cli backup-restore snapshot-restore list -n ${PROJECT_CPD_INSTANCE}

If you are using one of the following services, do the appropriate post-restore steps.

Service Post processing steps

Analytics Engine Powered by Apache Spark Clean up orphaned jobs and kernels.

Service	Post processing steps
Analytics Engine Powered by Apache Spark	Clean up orphaned jobs and kernels.
Watson Studio	If the Hadoop assembly is installed and you are restoring on a different cluster, you must reregister the Hadoop systems. Any jobs or interactive environments that were running at the time of backing up Watson Studio will not be automatically started after the restore. Scheduled jobs will also not automatically be rescheduled. You must edit each of those scheduled jobs to reactivate them. You should first remove the current schedule and then recreate that schedule to reactive it.
Watson Machine Learning	After restoring from a backup, users might be unable to deploy new models and score existing models. To resolve this issue, after the restore operation, wait until operator reconciliation completes. You can check the status of the operator with the following command: `kubectl describe WmlBase wml-cr -n <namespace_of_wml> \| grep "Wml Status" \| awk '{print $3}'`

Watson Studio

If the Hadoop assembly is installed and you are restoring on a different cluster, you must reregister the Hadoop systems.

Any jobs or interactive environments that were running at the time of backing up Watson Studio will not be automatically started after the restore. Scheduled jobs will also not automatically be rescheduled.

You must edit each of those scheduled jobs to reactivate them. You should first remove the current schedule and then recreate that schedule to reactive it.

Watson Machine Learning

After restoring from a backup, users might be unable to deploy new models and score existing models. To resolve this issue, after the restore operation, wait until operator reconciliation completes. You can check the status of the operator with the following command:

kubectl describe WmlBase wml-cr -n <namespace_of_wml> | grep "Wml Status" | awk '{print $3}'