Obtaining metrics to reflect bucket replication state

Learn how to obtain bucket-level replication metrics to evaluate replication progress.

About this task

To determine if the data is safe or available on the secondary site, the replication progress per bucket is provided using the following metrics:

  • Total number of objects scanned in the last replication cycle per bucket - bucket_last_cycle_total_objects_num{bucket="bucket-name"}
  • Number of objects successfully replicated in the last cycle per bucket - bucket_last_cycle_replicated_objects_num{bucket="bucket-name"}
  • Number of objects that failed replication in the last cycle per bucket - bucket_last_cycle_error_objects_num{bucket="bucket-name"}

Procedure

  1. Use the following commands to obtain the metrics:
    JWT_TOKEN=$(oc get secret noobaa-metrics-auth-secret -n openshift-storage -o jsonpath=" {.data.metrics_token}
    " | base64 -d)
    
    oc -n openshift-storage exec noobaa-core-0 – curl -k -H "Authorization: Bearer ${JWT_TOKEN}" localhost:8080/metrics/bg_workers
    For example:
    noobaa-operator$ oc -n openshift-storage exec -it noobaa-core-0 -- curl -k -H "Authorization: Bearer ${JWT_TOKEN}" http://localhost:8080/metrics/bg_workers | grep -E "bucket_name"| grep -v "NooBaa_replication_status"
    Defaulted container "core" out of: core, noobaa-log-processor
    
    NooBaa_bucket_last_cycle_total_objects_num{bucket_name="test-buck1"} 15
    NooBaa_bucket_last_cycle_total_objects_num{bucket_name="test-buck2"} 25
    NooBaa_bucket_last_cycle_replicated_objects_num{bucket_name="test-buck1"} 15
    NooBaa_bucket_last_cycle_replicated_objects_num{bucket_name="test-buck2"} 25
    NooBaa_bucket_last_cycle_error_objects_num{bucket_name="test-buck1"} 0
    NooBaa_bucket_last_cycle_error_objects_num{bucket_name="test-buck2"} 0
  2. Use the following syntax to view the metrics in the metrics section:
    metric_name {bucket_name='bucket_name'}

    For example:

    For NooBaa_bucket_last_cycle_total_objects_num

    NooBaa_bucket_last_cycle_total_objects_num {bucket_name='bubck1'}

    For NooBaa_bucket_last_cycle_replicated_objects_num

    NooBaa_bucket_last_cycle_replicated_objects_num {bucket_name='bubck1'}

    For NooBaa_bucket_last_cycle_error_objects_num

    NooBaa_bucket_last_cycle_error_objects_num {bucket_name='bubck1'}