Table of contents

Backing up the Cloud Pak for Data file system to a local repository or object store

You can do an offline backup of all persistent volumes (PVs) in your IBM® Cloud Pak for Data project to a local repository or an S3 or S3 compatible object store.

Before you begin

Cloud Pak for Data provides the cpd-cli backup-restore command-line interface for backing up and restoring the file system. Before you run any cpd-cli backup-restore commands, ensure that you completed the steps in Installing the Cloud Pak for Data file system backup and restore service.

About this task

The cpd-cli backup-restore command-line interface backs up and restores volume data in the same project and installation, and assumes that Kubernetes objects are still in place.

Important: Backing up persistent volumes alone is not sufficient for disaster recovery purposes, because Kubernetes objects like secrets are needed along with volume data to restore applications in a project.

During the backup process, write operations in application workloads are suspended (quiesced) so that you can do backups or other maintenance activities. The quiesce command calls hooks provided by Cloud Pak for Data services to do the quiesce. Quiesce hooks that are provided by Cloud Pak for Data services offer optimizations or other enhancements compared to scaling down all resources in the project (namespace). Services might be quiesced and unquiesced in a certain order, or services might be suspended without having to bring down pods to reduce the time it takes to bring down applications and bring them back up.

You can back up volumes in two ways:
  1. Manually scale down resources, back up volumes, and then manually scale up resources.
  2. Automatically scale down resources, back up volumes, and automatically scale up resources with a single command.
Tip: It is a good idea to manually scale down application Kubernetes resources before you do a backup, so that you can find out whether a pod's services cannot scale down correctly. You can then do the backup after you fix any problems that were found.

The commands in the following section use an example Cloud Pak for Data project named zen.

For a list of all possible options, enter the command ./cpd-cli backup-restore -h.

Procedure

  1. Initialize cpd-cli backup-restore.
    Note: If your Docker image registry is different than what is shown in the following examples, change the appropriate options.
    • OpenShift® 4.5 example:
      # Initialize the cpdbr first with pvc name and s3 storage.  Note that the bucket must exist.
      $ cpd-cli backup-restore init --namespace zen --pvc-name cpdbr-pvc --image-prefix=image-registry.openshift-image-registry.svc:5000/$NAMESPACE \
           --provider=s3 --s3-endpoint="s3 endpoint" --s3-bucket=cpdbr --s3-prefix=zen/
    • OpenShift 3.11 example:
      # Initialize the cpdbr first with pvc name and s3 storage.  Note that the bucket must exist.
      $ cpd-cli backup-restore init -n zen --pvc-name cpdbr-pvc --image-prefix=docker-registry.default.svc:5000/$NAMESPACE \
           --provider=s3 --s3-endpoint="s3 endpoint" --s3-bucket=cpdbr --s3-prefix=zen/
  2. To back up volumes by manually scaling down resources, backing up volumes, and manually scaling up resources, do the following steps.
    1. Manually scale down application Kubernetes resources.
      cpd-cli backup-restore quiesce -n zen

      If you want to ignore quiesce hooks, include the --ignore-hooks option.

    2. If you are backing up IBM Streams, delete the Streams application and build resources by running the following commands.
      oc delete pod -l streams-builder=active
      oc delete pod -l streams-app=active
    3. Check for completed jobs and pods by running the volume backup command with the --dry-run option, specifying a backup name identifier.

      The --dry-run option reports jobs or pods that are still attached to the PVCs to be backed up.

      Note: The backup name identifier must consist of lower case alphanumeric characters or the hyphen (-), and must start and end with an alphanumeric character. The underscore character (_) is not supported.
      cpd-cli backup-restore volume-backup create -n zen --dry-run backup_name
    4. If the dry run reports completed or failed jobs, or pods, that reference PVCs, delete them.
      Tip: Consider saving the job/pod yaml before you manually delete them, or include the --cleanup-completed-resources option in the backup step.
    5. Run the backup command with the --skip-quiesce option.
      cpd-cli backup-restore volume-backup create -n zen <backup_name_identifier> --skip-quiesce=true
      Notes:

      If you run another backup job with the same backup name, an incremental backup occurs. If you specify a new backup name, a full volume backup occurs.

      With certain storage providers, Kubernetes resources must be scaled down to unmount the PVCs before you create the backup. In such scenarios, the volume-backup create command with the --skip-quiesce option can fail if pods are running with mounted PVCs. If this problem occurs, use the quiesce command with the --ignore-hooks option to scale down the resources, and rerun the volume-backup create command with the --skip-quiesce option. You can then scale up the Kubernetes resources after backup by using the unquiesce command with the --ignore-hooks option.

    6. Manually scale up application Kubernetes resources.
      cpd-cli backup-restore unquiesce -n zen

      If you want to ignore quiesce hooks, include the --ignore-hooks option.

  3. To automatically scale down resources, back up volumes, and automatically scale up resources, do the following steps.
    Note: With auto-quiesce and auto-unquiesce, resources that are associated with PVCs are scaled down and up. Quiesce and unquiesce hooks are not invoked.
    1. If you are backing up IBM Streams, delete the Streams application and build resources by running the following commands.
      oc delete pod -l streams-builder=active
      oc delete pod -l streams-app=active
    2. Run the following volume backup command, specifying a backup name identifier.
      Note: The backup name identifier must consist of lower case alphanumeric characters or the hyphen (-), and must start and end with an alphanumeric character. The underscore character (_) is not supported.
      cpd-cli backup-restore volume-backup create -n zen <backup_name_identifier>
      Note: When you initiate a new backup with the same backup name identifier, only the incremental changes are updated to this backup.
    3. If the backup fails because there are completed or failed jobs, or pods, that reference PVCs, delete them, and rerun the backup command.
      Tip: Consider saving the job/pod yaml before you manually delete them, or include the --cleanup-completed-resources option in the backup command.
    4. If the backup does not automatically scale up resources because of a previous failure, manually scale up resources.
      cpd-cli backup-restore unquiesce -n zen --ignore-hooks
  4. To check the status of a backup job, run the following command:
    cpd-cli backup-restore volume-backup status -n zen Backup_name
  5. To view a list of existing volume backups, run the following command:
    cpd-cli backup-restore volume-backup list -n zen
  6. To view the size of the backup, running the following command:
    cpd-cli backup-restore volume-backup list --details -n zen
  7. To get the logs of a volume backup, run the following command:
    cpd-cli backup-restore volume-backup logs -n zen Backup_name
  8. Optional: After the volume backup is complete, clean up cpd-cli backup-restore (delete the cpd-cli backup-restore deployment and other metadata) by running the following command:
    cpd-cli backup-restore reset -n zen --force