Workflow

Edit online

A high-level workflow of On-prem Object Expiration includes,

  • User configures Object Expiration on a storage pool and then on the vaults on that storage pool.
  • User adds lifecycle policies to any or all buckets that expiration is enabled.
  • Each day, at midnight UTC, the scanning services and the reclamation service begin scanning for objects to be deleted. These services run on each access pool that expiration-enabled for a vault is deployed. The background work is shared between all Accessers in a pool.
    • The scanning services find objects to delete the following day. These services update a queue of objects to reclaim or a queue of incomplete multipart transactions to abort, which is evaluated by the reclamation service the following day. The services also communicate among themselves by updating queues that the following service reads.

      • The lifecycle-container-listing service lists all containers that have a lifecycle policy and passes these to the container-scanning-range service or the version-scanning-range service or the mpu-scanning service.

      • The container-scanning-range service or version-scanning-range service divides the objects in each bucket to smaller ranges and passes these ranges to the lifecycle-name-index-scan or version-scanning-range services.

      • The lifecycle-name-index-scan service or version-index-scan service reads the objects in each scanning range and puts the objects that, according to the bucket’s lifecycle policy, should be deleted the following day from the reclamation queue.

      • The mpu-scanning scanning service evaluates all the incomplete multipart transactions and determines based on the bucket’s lifecycle policy which ones should be aborted the following day and add them to the mpu-reclamation queue.

    • The reclamation service deletes objects that are due to expire on the current day. The service reads objects in the reclamation queue that were found the previous day, reevaluates the policy to make sure they’re still eligible for deletion, and then deletes them.

    • The mpu-reclamation service abort all the expired incomplete multipart uploads and deletes all the corresponding parts.