Viewing the performance counters for users and bucket

Edit online

The Ceph Object Gateway uses the performance counters to track metrics. You can visualize a cluster-wide view of the usage data over time in the Ceph Exporter port, which is usually, 9926, which includes PUT operations for objects in a bucket.

Before you begin

  1. A running IBM Storage Ceph cluster with Ceph Object Gateway installed.
  2. Monitoring stack enabled which includes Prometheus and ceph-exporter.

About this task

To track the operation metrics by users, set the rgw_user_counters_cache to true and to track the operation metrics by buckets, set the rgw_bucket_counters_cache to true.

You can use both rgw_user_counters_cache_size and rgw_bucket_counters_cache_size to set the number of entries in each cache.

Counters are evicted from a cache once the number of counters in the cache are greater than the cache size configuration variable. The counters that are evicted are the least recently used (LRU).

For example, if the number of buckets exceeded rgw_bucket_counters_cache_size by 1 and the counters with label bucket1 were the last to be updated, the counters for bucket1 get evicted from the cache. If S3 operations tracked by the operation metrics were done on bucket1 after eviction, all the metrics in the cache for bucket1 start at 0.

Cache sizing can depend on several factors, which include the following:

  • Number of users in the cluster.

  • Number of buckets in the cluster.

  • Memory usage of the Ceph Object Gateway.

  • Disk and memory usage of Prometheus.

  • To help calculate the Ceph Object Gateway’s memory usage of a cache, it should be noted that each cache entry, encompassing all the operation metrics, is 1360 bytes. This value is an estimate and subject to change if metrics are added or removed from the operation metrics list.

Important:

Since the operation metrics are labeled as performance counters, they live in memory. If the Ceph Object Gateway is restarted or crashes, all counters in the Ceph Object Gateway, whether in a cache or not, are lost.

Procedure

  1. Set the performance counters for users and buckets.
    1. Set the performance counter for users.

      For example,

      [ceph: root@host01 /]# ceph config set client.rgw.rgw.1.host04 rgw_user_counters_cache true
    2. Set the performance counter for buckets.
      For example, 
      [ceph: root@host01 /]# ceph config set client.rgw.rgw.1.host04 rgw_bucket_counters_cache true
    3. Restart the Ceph Object Gateway service.
      [ceph: root@host01 /]# ceph orch restart rgw.rgw.1
  2. Create users. For more information, see User management.
  3. Create buckets and upload objects into the bucket.
    1. Configure s3cmd.
      [root@host01 ~]# s3cmd --configure
    2. Create the S3 bucket.
      s3cmd mb s3://NAME_OF_THE_BUCKET_FOR_S3
      For example, 
      [root@host01 ~]# s3cmd mb s3://bucket
Bucket 's3://bucket/' created
    3. Create your file, input all the data, upload buckets on S3.
      s3cmd put FILE_NAME s3://NAME_OF_THE_BUCKET_FOR_S3
      For example, 
      [root@host01 ~]# s3cmd put test.txt s3://bucket

upload: 'test.txt' -> 's3://bucket/test.txt'  [1 of 1]
 21 of 21   100% in    1s    16.75 B/s  done
    4. Verify that the objects are uploaded.
      config dump ceph daemon DAEMON_ID counter dump
  4. Verify that the metrics are running on the local host.
    http://RGW_IP_ADDRESS:CEPH-EXPORTER_PORT
    Example for per bucket perf counter:
    # HELP ceph_rgw_op_per_bucket_put_obj_ops Puts
# TYPE ceph_rgw_op_per_bucket_put_obj_ops counter
ceph_rgw_op_per_bucket_put_obj_ops{bucket="test-bkt1",instance_id="ceph-ck-perf-ej61qj-node5"} 10
    Example for per user perf counter:
    # HELP ceph_rgw_op_per_user_put_obj_ops Puts
# TYPE ceph_rgw_op_per_user_put_obj_ops counter
ceph_rgw_op_per_user_put_obj_ops{instance_id="ceph-ck-perf-ej61qj-node5",user="ckulal"} 10
  5. Verify the metrics on Prometheus.
    https://RGW_IP_ADDRESS:PROMETHEUS_PORT
    For example, 
    https://10.0.210.100:9283/