Backup & Restore service architecture

This topic explains the high-level workings of the Backup & Restore service to replicate data. It provides an overall idea of the internal components and their roles in the backup and restore operation, resulting in the final transfer of data to designated storage.

The following diagram depicts the workflow of a Backup & Restore hub and local agent.

The IBM Fusion CRs (Application and Data Protections CRs) and operators (Application and Fusion DP operator) interact with the Backup & Restore service. The Backup & Restore service includes Agent and Server components.

Agents handle specific tasks to perform up to ten backup or restore jobs at once:
  • The Transaction Manager facilitates the backup or restore job requests from the server. These actions include request for PVC snapshots, recipe processing, and status updates to the server.
  • To process the backup, the Data Mover mounts the PVC snapshot to read the data, determines any deduplication, and then transfers the data to the designated S3 bucket.
The server includes components like Backup Policy, Backup Location, Job Manager, Application Service, Backup Service, Redis, and MongoDB.
  • The Backup Policy manages scheduling and cleanup tasks through Policy CRs.
  • the Backup Location validates BSL CRs without performing extensive validations as it directly communicates with the S3 bucket to verify connectivity and existence.
  • The Job Manager distributes jobs to agents and monitors job progress to detect and cancel hung jobs. It also prevents duplicate jobs for the same application or policy combination from running at the same time.
  • Applications Service maintains records of all existing applications across clusters.
  • The Backup Service updates backup and restore Custom Resource with their respective statuses.
  • Redis is employed to manage lock contentions; for instance, during scheduled backups involving replica instances, after one acquires a Redis lock, it initiates the task while others are dormant.
  • MongoDB functions as the central repository for backup and metadata details.

Backup & Restore service uses Kafka as the primary communication medium among services, providing scalability and resilience. It also allows multiple replicas to process distinct messages concurrently.

The following diagram depicts the workflow of a Backup & Restore hub with multiple spokes.

In case of hub and spoke, it is similar but the agents are on different clusters. The Fusion UI, CRs, Backup & Restore server and local agent are the same except that it needs additional cross cluster communication.

When you attach a spoke cluster to the hub, install the Backup & Restore Agent service on the spoke. The installation of the Backup & Restore Agent service requires a temporary token provided by the Hub for initial setup of the secure cross-cluster communication between the hub and spoke.