Performing localized reads

Run a localized read on a replicated pool in a stretch cluster.

Before you begin

Before you begin, make sure that you have the following prerequisites in place:

A stretch cluster with two data centers and a Ceph Object Gateway that is configured on both.
A user who is created with a bucket having primary and replicated OSDs.

About this task

When a localized read request is made on a replicated pool, Ceph selects the local OSDs closest to the client based on the client location specified in crush_location.

Procedure

To run a localized read, set rados_replica_read_policy to localize in the OSD daemon configuration by using the ceph config set command.

ceph config set client.rgw.rgw.1 rados_replica_read_policy localize

What to do next

Verify the localized read from an OSD set.

Run the ceph osd tree command to view the OSDs and the data centers.

For example,

[ceph: root@host01 /]# ceph osd tree

ID  CLASS  WEIGHT   TYPE NAME                                 STATUS  REWEIGHT  PRI-AFF
-1         0.58557  root default
-3         0.29279      datacenter DC1
-2         0.09760          host ceph-ci-fbv67y-ammmck-node2
 2    hdd  0.02440              osd.2                             up   1.00000  1.00000
11    hdd  0.02440              osd.11                            up   1.00000  1.00000
17    hdd  0.02440              osd.17                            up   1.00000  1.00000
22    hdd  0.02440              osd.22                            up   1.00000  1.00000
-4         0.09760          host ceph-ci-fbv67y-ammmck-node3
 0    hdd  0.02440              osd.0                             up   1.00000  1.00000
 6    hdd  0.02440              osd.6                             up   1.00000  1.00000
12    hdd  0.02440              osd.12                            up   1.00000  1.00000
18    hdd  0.02440              osd.18                            up   1.00000  1.00000
-5         0.09760          host ceph-ci-fbv67y-ammmck-node4
 5    hdd  0.02440              osd.5                             up   1.00000  1.00000
10    hdd  0.02440              osd.10                            up   1.00000  1.00000
16    hdd  0.02440              osd.16                            up   1.00000  1.00000
23    hdd  0.02440              osd.23                            up   1.00000  1.00000
-7         0.29279      datacenter DC2
-6         0.09760          host ceph-ci-fbv67y-ammmck-node5
 3    hdd  0.02440              osd.3                             up   1.00000  1.00000
 8    hdd  0.02440              osd.8                             up   1.00000  1.00000
14    hdd  0.02440              osd.14                            up   1.00000  1.00000
20    hdd  0.02440              osd.20                            up   1.00000  1.00000
-8         0.09760          host ceph-ci-fbv67y-ammmck-node6
 4    hdd  0.02440              osd.4                             up   1.00000  1.00000
 9    hdd  0.02440              osd.9                             up   1.00000  1.00000
15    hdd  0.02440              osd.15                            up   1.00000  1.00000
21    hdd  0.02440              osd.21                            up   1.00000  1.00000
-9         0.09760          host ceph-ci-fbv67y-ammmck-node7
 1    hdd  0.02440              osd.1                             up   1.00000  1.00000
 7    hdd  0.02440              osd.7                             up   1.00000  1.00000
13    hdd  0.02440              osd.13                            up   1.00000  1.00000
19    hdd  0.02440              osd.19                            up   1.00000  1.00000

Run the ceph orch command to identify the Ceph Object Gateway daemons in the data centers.

For example,

[ceph: root@host01 /]# ceph orch ps | grep rg

rgw.rgw.1.ceph-ci-fbv67y-ammmck-node4.dmsmex         ceph-ci-fbv67y-ammmck-node4            *:80              running (4h)     10m ago  22h    93.3M        -  19.1.0-55.el9cp  0ee0a0ad94c7  34f27723ccd2
rgw.rgw.1.ceph-ci-fbv67y-ammmck-node7.pocecp         ceph-ci-fbv67y-ammmck-node7            *:80              running (4h)     10m ago  22h    96.4M        -  19.1.0-55.el9cp  0ee0a0ad94c7  40e4f2a6d4c4

Verify if the localized read took place by running the vim command on the Ceph Object Gateway logs.

vim /var/log/ceph/<fsid>/<ceph-client-rgw>.log

For example,

[ceph: root@host01 /]# vim /var/log/ceph/<fsid>/<ceph-client-rgw>.log

2024-08-26T08:07:45.471+0000 7fc623e63640  1 ====== starting new request req=0x7fc5b93694a0 =====
2024-08-26T08:07:45.471+0000 7fc623e63640  1 -- 10.0.67.142:0/279982082 --> [v2:10.0.66.23:6816/73244434,v1:10.0.66.23:6817/73244434] -- osd_op(unknown.0.0:9081 11.55 11:ab26b168:::3acf4091-c54c-43b5-a495-c505fe545d25.27842.1_f1:head [getxattrs,stat] snapc 0=[] ondisk+read+localize_reads+known_if_redirected+supports_pool_eio e3533) -- 0x55f781bd2000 con 0x55f77f0e8c00

You can see in the logs that a localized read has taken place.

Important: To be able to view the debug logs, you must first enable debug_ms 1 in the configuration by running the ceph config set command.

For example,

[ceph: root@host01 /]#ceph config set client.rgw.rgw.1.ceph-ci-gune2w-mysx73-node4.dgvrmx    advanced  debug_ms    1/1

[ceph: root@host01 /]#ceph config set client.rgw.rgw.1.ceph-ci-gune2w-mysx73-node7.rfkqqq    advanced  debug_ms    1/1