Lifecycle rules

Lifecycle rules specify when an Amazon S3 file is set to expire or be archived.

Types of lifecycle rules

In the Amazon S3 Connector stage, you can specify the following types of lifecycle rules:
Expiration
When a file expires, it is deleted from Amazon Web Services.
Archival
When a file is archived, it is sent to the Amazon Glacier archive.

For both types of lifecycle rules, you can specify the date when the lifecycle rule is set to apply or the number of days that the file will exist in its current state before the lifecycle rule is set to apply. If you define lifecycle rules for archival and expiration for a file or folder, both lifecycle rules must use the same time period format.

Rule scope

A lifecycle rule can apply to only one file or to all of the files in the folder that contains the file. You cannot specify a lifecycle rule that applies to a file and a lifecycle rule that applies to the folder that contains the file.

When lifecycle rules are created in Amazon S3, the scope of the rule is represented by a prefix. This prefix includes only the part of the file path for a file that the lifecycle rule applies to.

For example, suppose that you want to create a lifecycle rule for the directory4/file1.txt file. If you create a lifecycle rule and set the Rule scope property to File, the prefix is directory4/file1.txt. If you create a lifecycle rule and set the Rule scope property to Folder, the prefix is directory4/.

If the connector runs in parallel, the lifecycle rule must apply to all of the files in the folder. As a result, the Rule scope property must be set to Folder.

Examples

Suppose that on 1 January 2016, you want to create a lifecycle rule for all of the files in the DepartmentA/Employees folder. You want the files to be archived in three years and deleted in five years. Because you can specify lifecycle rules based on a date or a number of days but you must use the same format for both rules, you can configure the lifecycle rules in the following ways:

Table 1. Example of options for specifying lifecycle rules
Amazon S3 property Value for specifying lifecycle rules based on a date Value for specifying lifecycle rules based on a number of days
Rule scope Folder Folder
Time period format Specific date Days from creation date
Expiration Yes Yes
Expiration date "2021-01-01" Does not apply
Duration Does not apply 1825
Archive Yes Yes
Date to archive "2019-01-01" Does not apply
Duration Does not apply 1095