Restoring an offline backup to the same cluster with the Cloud Pak for Data OADP backup and restore utility

Important: IBM Cloud Pak® for Data Version 4.7 will reach end of support (EOS) on 31 July, 2025. For more information, see the Discontinuance of service announcement for IBM Cloud Pak for Data Version 4.X.

Upgrade to IBM Software Hub Version 5.1 before IBM Cloud Pak for Data Version 4.7 reaches end of support. For more information, see Upgrading IBM Software Hub in the IBM Software Hub Version 5.1 documentation.

You can restore an IBM Cloud Pak for Data instance from an offline backup on the same cluster with the OADP backup and restore utility.

About this task

Permissions that you need for this task: Log on as a user with cluster administrator rights.

You cannot restore a backup into a different project of the Cloud Pak for Data instance.

To restore a backup, you must run the Cloud Pak for Data OADP backup and restore utility in Kubernetes mode.

If running a restore command produces a Failed or PartiallyFailed error, you must clean up the Cloud Pak for Data instance and restart the restore process. For details about cleaning up a Cloud Pak for Data instance, see Preparing to restore Cloud Pak for Data.

Best practice: You can run the commands in this task exactly as written if you set up environment variables. For instructions, see Setting up installation environment variables.

Ensure that you source the environment variables before you run the commands in this task.

Procedure

Check for problems that will prevent a successful restore:
```
cpd-cli oadp restore precheck \
--backup-names=<backup_name1,backup_name2>
```
This command checks for problems such as:
- The OADP project does not exist.
- A Velero instance is not running or its pod is not healthy.
- The OADP backup storage location custom resource is not in the Available state.
- The specified backup name does not exist.
- The backup did not complete successfully or has errors or warnings.
- Projects that are associated with the backup exist.
For the complete list of checks that the command performs, see oadp restore precheck.

Restore Kubernetes resource definitions, projects, OperatorGroups, and permissions, providing the Kubernetes resources backup name and specifying a restore name:

The cluster pulls images from the IBM Entitled Registry

Restriction: This option is available only if the cluster can connect to the internet.

cpd-cli oadp restore create <restore_name1> \
--from-backup=<backup_name1> \
--include-resources='namespaces,operatorgroups,roles,rolebindings,serviceaccounts,customresourcedefinitions.apiextensions.k8s.io,securitycontextconstraints.security.openshift.io' \
--include-cluster-resources=true \
--image-prefix=registry.redhat.io/ubi8 \
--skip-hooks \
--log-level=debug \
--verbose

The cluster pulls images from a private container registry

Restriction: This option is available only if an administrator moved the backup and restore images to the private container registry. For details, see Moving images for the backup and restore utilities to the private container registry.

cpd-cli oadp restore create <restore_name1> \
--from-backup=<backup_name1> \
--include-resources='namespaces,operatorgroups,roles,rolebindings,serviceaccounts,customresourcedefinitions.apiextensions.k8s.io,securitycontextconstraints.security.openshift.io' \
--include-cluster-resources=true \
--image-prefix=${PRIVATE_REGISTRY_LOCATION}/ubi8 \
--skip-hooks \
--log-level=debug \
--verbose

Tip: If you are restoring Db2® or Watson Knowledge Catalog, add

 --scale-wait-timeout
15m

to ensure that the restore command completes successfully.

Restore Kubernetes CommonService custom resources, providing the Kubernetes resources backup name and specifying a restore name:

The cluster pulls images from the IBM Entitled Registry

Restriction: This option is available only if the cluster can connect to the internet.

cpd-cli oadp restore create <restore_name2> \
--from-backup=<backup_name1> \
--include-resources='namespacescopes,commonservices' \
--include-cluster-resources=true \
--image-prefix=registry.redhat.io/ubi8 \
--skip-hooks \
--log-level=debug \
--verbose

The cluster pulls images from a private container registry

cpd-cli oadp restore create <restore_name2> \
--from-backup=<backup_name1> \
--include-resources='namespacescopes,commonservices' \
--include-cluster-resources=true \
--image-prefix=${PRIVATE_REGISTRY_LOCATION}/ubi8 \
--skip-hooks \
--log-level=debug \
--verbose

Note: If this command results in a PartiallyFailed restore status with an error such as in the following example, the error indicates that you are attempting the restore on a cluster where the Cloud Pak for Data operators visibility to the Cloud Pak for Data operand projects is not yet completed.

Errors:
  Velero:     <none>
  Cluster:    <none>
  Namespaces:
    cpd-instance:  error restoring commonservices.operator.ibm.com/cpd-instance/common-service: admission webhook "ibm-common-service-validating-webhook.operator.ibm.com" denied the request: Services Namespace: cpd-instance is not allowed to be configured in namespace cpd-instance

Restore the Kubernetes cpd-operators configmap resource, providing the Kubernetes resources backup name and specifying a restore name:
The cluster pulls images from the IBM Entitled Registry

Restriction: This option is available only if the cluster can connect to the internet.
```
cpd-cli oadp restore create <restore_name3> \
--from-backup=<backup_name1> \
--include-resources='configmaps' \
--selector 'app=cpd-operators-backup' \
--image-prefix=registry.redhat.io/ubi8 \
--skip-hooks \
--log-level=debug \
--verbose
```
The cluster pulls images from a private container registry

Restriction: This option is available only if an administrator moved the backup and restore images to the private container registry. For details, see Moving images for the backup and restore utilities to the private container registry.
```
cpd-cli oadp restore create <restore_name3> \
--from-backup=<backup_name1> \
--include-resources='configmaps' \
--selector 'app=cpd-operators-backup' \
--image-prefix=${PRIVATE_REGISTRY_LOCATION}/ubi8 \
--skip-hooks \
--log-level=debug \
--verbose
```
Restore (recreate) operators:
```
./cpd-operators.sh restore \
--foundation-namespace ${PROJECT_CPD_INST_OPERATORS} \
--operators-namespace ${PROJECT_CPD_INST_OPERATORS}
```
Note: IBM Cloud Pak foundational services operators are installed in the same namespace as Cloud Pak for Data operators.
This step restores the Operator CatalogSource, Subscription, OperandRequest, and NamespaceScope custom resources, which must be validated before you can continue with the restore process.
Validate operator subscriptions and visibility.
1. Query the status of the custom resources in the Cloud Pak for Data operator and operand projects:
```
oc get sub -n ${PROJECT_CPD_INST_OPERATORS} -o jsonpath="{range .items[*]}{.metadata.name}{' - currentCSV: '}{.status.currentCSV}{' - installedCSV: '}{.status.installedCSV}{'\n'}{end}"
oc get operandrequest -A
```
2. Validate the restore of the Cloud Pak for Data operators visibility:
```
oc get nss -n ${PROJECT_CPD_INST_OPERATORS} common-service -o jsonpath='{.status.validatedMembers} {"\n"}' 
```
  The output of this command must be an array that contains the ${PROJECT_CPD_INST_OPERATORS} and all operand projects.

Restore Kubernetes ZenService resources, providing the Kubernetes resources backup name and specifying a restore name:

The cluster pulls images from the IBM Entitled Registry

Restriction: This option is available only if the cluster can connect to the internet.

cpd-cli oadp restore create <restore_name4> \
--from-backup=<backup_name2> \
--include-resources='namespaces,secrets,configmaps,certificates.cert-manager.io,certificates.certmanager.k8s.io,issuers.cert-manager.io,issuers.certmanager.k8s.io,zenservices' \
--image-prefix=registry.redhat.io/ubi8 \
--skip-hooks \
--log-level=debug \
--verbose

The cluster pulls images from a private container registry

cpd-cli oadp restore create <restore_name4> \
--from-backup=<backup_name2> \
--include-resources='namespaces,secrets,configmaps,certificates.cert-manager.io,certificates.certmanager.k8s.io,issuers.cert-manager.io,issuers.certmanager.k8s.io,zenservices' \
--image-prefix=${PRIVATE_REGISTRY_LOCATION}/ubi8 \
--skip-hooks \
--log-level=debug \
--verbose

Restore the remaining Kubernetes resources (excluding EDB Postgres cluster resources), providing the Kubernetes resources backup name and specifying a restore name:

The cluster pulls images from the IBM Entitled Registry

Restriction: This option is available only if the cluster can connect to the internet.

cpd-cli oadp restore create <restore_name5> \
--from-backup=<backup_name2> \
--exclude-resources='secrets,configmaps,certificates.cert-manager.io,certificates.certmanager.k8s.io,issuers.cert-manager.io,issuers.certmanager.k8s.io,zenservices' \
--include-cluster-resources=true \
--image-prefix=registry.redhat.io/ubi8 \
--skip-hooks \
--log-level=debug \
--verbose

The cluster pulls images from a private container registry

cpd-cli oadp restore create <restore_name5> \
--from-backup=<backup_name2> \
--exclude-resources='secrets,configmaps,certificates.cert-manager.io,certificates.certmanager.k8s.io,issuers.cert-manager.io,issuers.certmanager.k8s.io,zenservices' \
--include-cluster-resources=true \
--image-prefix=${PRIVATE_REGISTRY_LOCATION}/ubi8 \
--skip-hooks \
--log-level=debug \
--verbose

Delete fenced off EDB Postgres pods:

oc delete po -l k8s.enterprisedb.io/cluster -n ${PROJECT_CPD_INST_OPERANDS}

Show the list of pods that were restored from the backups:

oc get pods -n ${PROJECT_CPD_INST_OPERANDS} -o=json | jq -r '.items[] | select(.metadata.annotations | has("cpdbr.cpd.ibm.com/backup-server-version"))|.metadata.name'

From the list of pods, identify pods that have problems, such as pods that are in a Failed state, and delete them.
```
oc delete pods -n ${PROJECT_CPD_INST_OPERANDS} <pod-name>
```

Restore EDB Postgres cluster resources and execute hooks, providing the Kubernetes resources backup name and specifying a restore name:

The cluster pulls images from the IBM Entitled Registry

Restriction: This option is available only if the cluster can connect to the internet.

cpd-cli oadp restore create <restore_name6> \
--from-backup=<backup_name1> \
--include-resources='namespaces,clusters.postgresql.k8s.enterprisedb.io' \
--include-cluster-resources=true \
--posthooks=true \
--image-prefix=registry.redhat.io/ubi8 \
--log-level=debug \
--verbose

The cluster pulls images from a private container registry

cpd-cli oadp restore create <restore_name6> \
--from-backup=<backup_name1> \
--include-resources='namespaces,clusters.postgresql.k8s.enterprisedb.io' \
--include-cluster-resources=true \
--posthooks=true \
--image-prefix=${PRIVATE_REGISTRY_LOCATION}/ubi8 \
--log-level=debug \
--verbose

To check the status of a restore, run the following command:
```
cpd-cli oadp restore status <restore_name> \
--details
```
To view a list of existing restores, run the following command:
```
cpd-cli oadp restore list
```
To view restore logs, run the following command:
```
cpd-cli oadp restore logs <restore_name>
```
To delete a restore for cleanup purposes, run the following command:
```
cpd-cli oadp restore delete <restore_name>
```

What to do next

If your Cloud Pak for Data deployment has services that connect to an external database, and you followed the recommendation to back up the database at the same time that you back up Cloud Pak for Data, restore the database backup that was taken at the same time as the Cloud Pak for Data backup.