Synchronizing versions in Multicloud Object Gateway bucket replication

Learn how to synchronize versions in Multicloud Object Gateway (MCG) bucket replication.

Before you begin

A Multicloud Object Gateway (MCG) source bucket, which is created from an object bucket claim (OBC) and any MCG target bucket. For example, you can create the two buckets using OBCs using the MCG command line interface (CLI):

Create a source bucket using an OBC:
```
mcg-cli obc create source-bucket --exact
mcg-cli obc create target-bucket --exact
```
Where, --exact is optional.

Ensure that S3 client aliases with MCG credentials and endpoint are set up.

NOOBAA_ACCESS_KEY=$(oc extract secret/noobaa-admin -n openshift-storage --keys=AWS_ACCESS_KEY_ID --to=- 2>/dev/null); \
NOOBAA_SECRET_KEY=$(oc extract secret/noobaa-admin -n openshift-storage --keys=AWS_SECRET_ACCESS_KEY --to=- 2>/dev/null); \
S3_ENDPOINT=https://$(oc get route s3 -n openshift-storage -o json | jq -r ".spec.host")

alias common_s3='AWS_ACCESS_KEY_ID=$NOOBAA_ACCESS_KEY AWS_SECRET_ACCESS_KEY=$NOOBAA_SECRET_KEY aws --endpoint $S3_ENDPOINT --no-verify-ssl'; \
alias s3_alias='common_s3 s3'; \
alias s3api_alias='common_s3 s3api'

Make sure to enable versioning on both the source and target bucket by using the put-bucket-versioning command in the AWS S3 client:

s3api_alias put-bucket-versioning --bucket source-bucket --versioning-configuration Status=Enabled

s3api_alias put-bucket-versioning --bucket target-bucket --versioning-configuration Status=Enabled

Procedure

Patch or edit a replication policy with sync_versions: true to the source bucket’s OBC:
```
oc patch obc source-bucket -n openshift-storage --type=merge -p '{"spec": {"additionalConfig": {"replicationPolicy": "{\"rules\":[{\"rule_id\":\"replication-rule-a\",\"destination_bucket\":\"target-bucket\", \"sync_versions\": true}]}"}}}'
```
Normal bucket replication replicates only the latest version, and the use of sync_versions add the functionality of replicating even the older versions, and in their original order.
Note:
- Delete markers are replicated if configured and using log base replication.
- Delete version IDs are not replicated to avoid human errors.

Verify the replication to the target bucket by adding a few versions under an object key and waiting for them to get replicated to the target bucket:

echo 'version_a' | s3_alias cp - s3://source-bucket/versioned_obj.txt

echo 'version_b' | s3_alias cp - s3://source-bucket/versioned_obj.txt

echo 'version_c' | s3_alias cp - s3://source-bucket/versioned_obj.txt

After a few minutes, compare the versions of the object on both the buckets:

s3api_alias list-object-versions --bucket source-bucket --prefix versioned_obj.txt | jq -r ".Versions[].ETag"
"aaabf266d38a8e995cef03c13ee9a7f1"
"181d8a23f59939c0cddec1692e05cdf3"
"ebc538cc6ffa04a39263f4b1be2f832f"

s3api_alias list-object-versions --bucket target-bucket --prefix versioned_obj.txt | jq -r ".Versions[].ETag"
"aaabf266d38a8e995cef03c13ee9a7f1"
"181d8a23f59939c0cddec1692e05cdf3"
"ebc538cc6ffa04a39263f4b1be2f832f"