Creating an offline backup of Cloud Pak for Data with the OADP utility

You can create offline backups of a Cloud Pak for Data instance with the Cloud Pak for Data OpenShift® APIs for Data Protection (OADP) backup and restore utility.

Before you begin

To create Restic backups, if Cloud Pak for Data is installed on NFS, NFS storage must be configured with no_root_squash.

About this task

Permissions needed for this task
If you are running the utility in Kubernetes mode, log in as a user with cluster administrator rights.

If you are running the utility in REST mode, make sure that the REST client is configured so that a Cloud Pak for Data administrator can run backup and checkpoint commands.

Backups are taken at the Cloud Pak for Data instance (tenant) level.

When backup commands are run, some pods remain in a Running state. These running pods do not affect the backup process, and you do not need to manually shut them down.

Note: The storage provider that you use to store backups might limit the number of snapshots that you can take per volume. For more information, consult your storage provider documentation.

For more information about the OADP backup and restore utility, including a list of commands that you can run, see the cpd-cli oadp reference documentation.

Best practice: You can run the commands in this task exactly as written if you set up environment variables. For instructions, see Setting up installation environment variables.

Ensure that you source the environment variables before you run the commands in this task.

Procedure

  1. If you are running the backup and restore utility in Kubernetes mode, log in to Red Hat® OpenShift Container Platform as a cluster administrator:
    ${OC_LOGIN}
    Remember: OC_LOGIN is an alias for the oc login command.
  2. Ensure that the expected EDB Postgres replica PVCs are included in the backup:
    oc label pvc,pods -l k8s.enterprisedb.io/cluster,velero.io/exclude-from-backup=true velero.io/exclude-from-backup- -n ${PROJECT_CPD_INST_OPERANDS}
  3. Check that the primary instance of every EDB Postgres cluster is in sync with its replicas.
    1. Run the following command:
      cpd-cli oadp backup precheck \
      --tenant-operator-namespace=${PROJECT_CPD_INST_OPERATORS} \
      --verbose
    2. If the output of the command shows that a primary instance is not in sync with a replica, delete the replica and its associated PVC.

    For more information about the command, see oadp backup precheck.

  4. If you did not set up installation environment variables, set the environment variable for the Cloud Pak for Data version that you are using:
    export VERSION=<Cloud Pak for Data_version>
  5. Generate the cpd-operators ConfigMap.

    This ConfigMap captures the required Kubernetes objects that you will need when you restore the operators.

    ./cpd-operators.sh backup \
    --foundation-namespace ${PROJECT_CPD_INST_OPERATORS} \
    --operators-namespace ${PROJECT_CPD_INST_OPERATORS} \
    --backup-iam-data
    Note: IBM Cloud Pak foundational services operators are installed in the same namespace as Cloud Pak for Data operators.
  6. Create a backup of Kubernetes resources and volume data (excluding EDB Postgres clusters), and provide a name for the backup.
    Notes:
    • The following command creates a restic backup. If you want to create a CSI snapshot instead, delete --default-volumes-to-fs-backup \ and set --snapshot-volumes to true. To create CSI snapshots, a valid volume snapshot class must be created. For more information, see Creating volume snapshot classes.
    • You can adjust the options --vol-mnt-pod-mem-request and --vol-mnt-pod-mem-limit based on the available resources. These options impact the memory request and limit respectively of the cpdbr-vol-mnt pod, which is used for pod volume backups of non-excluded orphaned PVCs. These options apply only to restic backups.

    The cluster pulls images from the IBM Entitled Registry
    Restriction: This option is available only if the cluster can connect to the internet.
    cpd-cli oadp backup create <tenant-offline-b1> \
    --tenant-operator-namespace ${PROJECT_CPD_INST_OPERATORS} \
    --exclude-tenant-operator-namespace=true \
    --exclude-resources='event,event.events.k8s.io,imagetags.openshift.io,operatorgroups,roles,rolebindings,serviceaccounts,customresourcedefinitions.apiextensions.k8s.io,securitycontextconstraints.security.openshift.io,catalogsources.operators.coreos.com,subscriptions.operators.coreos.com,clusterserviceversions.operators.coreos.com,installplans.operators.coreos.com,operandconfig,operandregistry,operandrequest,clients.oidc.security.ibm.com,authentication.operator.ibm.com,namespacescopes,commonservices,clusters.postgresql.k8s.enterprisedb.io,certificaterequests.cert-manager.io,orders.acme.cert-manager.io,challenges.acme.cert-manager.io' \
    --default-volumes-to-restic \
    --snapshot-volumes=false \
    --cleanup-completed-resources \
    --vol-mnt-pod-mem-request=1Gi \
    --vol-mnt-pod-mem-limit=4Gi \
    --image-prefix=registry.redhat.io/ubi9 \
    --log-level=debug \
    --verbose

    The cluster pulls images from a private container registry
    Restriction: This option is available only if an administrator moved the backup and restore images to the private container registry. For details, see Moving images for the cpd-cli to the private container registry.
    cpd-cli oadp backup create <tenant-offline-b1> \
    --tenant-operator-namespace ${PROJECT_CPD_INST_OPERATORS} \
    --exclude-tenant-operator-namespace=true \
    --exclude-resources='event,event.events.k8s.io,imagetags.openshift.io,operatorgroups,roles,rolebindings,serviceaccounts,customresourcedefinitions.apiextensions.k8s.io,securitycontextconstraints.security.openshift.io,catalogsources.operators.coreos.com,subscriptions.operators.coreos.com,clusterserviceversions.operators.coreos.com,installplans.operators.coreos.com,operandconfig,operandregistry,operandrequest,clients.oidc.security.ibm.com,authentication.operator.ibm.com,namespacescopes,commonservices,clusters.postgresql.k8s.enterprisedb.io,certificaterequests.cert-manager.io,orders.acme.cert-manager.io,challenges.acme.cert-manager.io' \
    --default-volumes-to-restic \
    --snapshot-volumes=false \
    --cleanup-completed-resources \
    --vol-mnt-pod-mem-request=1Gi \
    --vol-mnt-pod-mem-limit=4Gi \
    --image-prefix=${PRIVATE_REGISTRY_LOCATION}/ubi9 \
    --log-level=debug \
    --verbose

  7. Create a backup of Cloud Pak for Data operators and EDB Postgres cluster resources, and provide a name for the backup.

    The --tenant-operator-namespace parameter specifies the instance (tenant) operator project, and will implicitly include other instance projects (control plane and any tethered projects). If you want to back up only specific projects, use the --include-namespaces parameter instead, and specify the list of projects that you want to back up.

    Note: The --tenant-operator-namespace and --include-namespaces parameters are mutually exclusive.

    The cluster pulls images from the IBM Entitled Registry
    Restriction: This option is available only if the cluster can connect to the internet.
    cpd-cli oadp backup create <tenant-offline-b2> \
    --tenant-operator-namespace ${PROJECT_CPD_INST_OPERATORS} \
    --include-resources='namespaces,operatorgroups,roles,rolebindings,serviceaccounts,customresourcedefinitions.apiextensions.k8s.io,securitycontextconstraints.security.openshift.io,configmaps,secrets,namespacescopes,commonservices,clusters.postgresql.k8s.enterprisedb.io,physicallocationsets.cluster.ibm-cpd-mcscheduler.ibm.com,distributionrules.cluster.ibm-cpd-mcscheduler.ibm.com' \
    --skip-hooks \
    --image-prefix=registry.redhat.io/ubi9 \
    --log-level=debug \
    --verbose

    The cluster pulls images from a private container registry
    Restriction: This option is available only if an administrator moved the backup and restore images to the private container registry. For details, see Moving images for the cpd-cli to the private container registry.
    cpd-cli oadp backup create <tenant-offline-b2> \
    --tenant-operator-namespace ${PROJECT_CPD_INST_OPERATORS} \
    --include-resources='namespaces,operatorgroups,roles,rolebindings,serviceaccounts,customresourcedefinitions.apiextensions.k8s.io,securitycontextconstraints.security.openshift.io,configmaps,secrets,namespacescopes,commonservices,clusters.postgresql.k8s.enterprisedb.io,physicallocationsets.cluster.ibm-cpd-mcscheduler.ibm.com,distributionrules.cluster.ibm-cpd-mcscheduler.ibm.com' \
    --skip-hooks \
    --image-prefix=${PRIVATE_REGISTRY_LOCATION}/ubi9 \
    --log-level=debug \
    --verbose

  8. Check the status of the backup:
    cpd-cli oadp backup status <backup-name> \
    --details

    In the output, check that the Phase is Completed, and that the Resource List contains a VolumeSnapshot for each PVC.

    If you specified to create a restic backup, a list of completed restic backups appears at the end of the output. For example:
    Restic Backups:
    
    Completed:
    
    cpd-instance/cpdbr-vol-mnt: export-zen-minio-0-vol, export-zen-minio-1-vol, export-zen-minio-2-vol, ibm-zen-cs-mongo-backup-vol, ibm-zen-objectstore-backup-pvc-vol
    
    cpd-instance/zen-metastore-edb-2: pgdata 
  9. To view a list of existing backups, run the following command:
    cpd-cli oadp backup list
  10. To view backup logs, run the following command:
    cpd-cli oadp backup logs <backup-name>
  11. To delete a backup for cleanup purposes, run the following command:
    cpd-cli oadp backup delete <backup-name>

What to do next

If a backup fails, you must return Cloud Pak for Data to a good state before you can retry a backup. Run the following command:
cpd-cli oadp backup posthooks --hook-kind=br --tenant-operator-namespace=${PROJECT_CPD_INST_OPERATORS}

If you have services that connect to an external database, such as for business intelligence (BI) reporting, it is recommended that you also back up the database. Backing up the external database ensures data consistency if the Cloud Pak for Data backup is later restored. For example, you need to restore an older Cloud Pak for Data backup instead of the most recent backup. The external database is synchronized with the most recent Cloud Pak for Data backup, so it has data that is not in the backup that you want to restore. To maintain data consistency, you need to restore the external database backup that was taken at the same time as the Cloud Pak for Data backup.