Adding D3N cache directory

Edit online

To enable D3N cache on Ceph Object Gateway, you need to also include the D3N cache directory in the podman unit.run file.

Before you begin

Before you begin, make sure that you have the following prerequisites in place:

A running IBM Storage Ceph cluster.
Ceph Object Gateway installed.
Root-level access to the admin node.
A fast NVMe drive in each Ceph Object Gateway node to serve as the local cache storage.

Procedure

Create a mount point for the NVMe drive.

mkfs.ext4 NVME_DRIVE_PATH

For example,

[ceph: root@host01 /]# mkfs.ext4 /dev/nvme0n1
mount /dev/nvme0n1 /mnt/nvme0n1/

Create a cache directory path.

mkdir NVME_MOUNT_PATH/CACHE_DIRECTORY_NAME

For example,

[ceph: root@host01 /]# mkdir /mnt/nvme0n1/rgw_datacache

Provide a+rwx permission to NVME_MOUNT_PATH and to the rgw_d3n_l1_datacache_persistent_path.

chmod a+rwx NVME_MOUNT_PATH ; chmod a+rwx rgw_d3n_l1_datacache_persistent_path

For example,

[ceph: root@host01 /]# chmod a+rwx /mnt/nvme0n1 ; chmod a+rwx /mnt/nvme0n1/rgw_datacache/

Create or modify the Ceph Object Gateway specification file with extra_container_args to add rgw_d3n_l1_datacache_persistent_path into podman unit.run file.

"extra_container_args:
          "-v"
          "rgw_d3n_l1_datacache_persistent_path:rgw_d3n_l1_datacache_persistent_path"
			"

For example,

[ceph: root@host01 /]# cat rgw-spec.yml
	service_type: rgw
	service_id: rgw.test
	placement:
		hosts:
		    host1
		    host2
		extra_container_args:
		    "-v"
		    "/mnt/nvme0n1/rgw_datacache/:/mnt/nvme0n1/rgw_datacache/"

If there are multiple instances of Ceph Object Gateway in a single host, then a separate rgw_d3n_l1_datacache_persistent_path must be created for each instance and add each path in extra_container_args.

For two instances of Ceph Object Gateway in each host, create two separate instances of cache-directory under rgw_d3n_l1_datacache_persistent_path, /mnt/nvme0n1/rgw_datacache/rgw1 and /mnt/nvme0n1/rgw_datacache/rgw2.

Example for extra_container_args in a Ceph Object Gateway specification file

"extra_container_args:
	"-v"
	"/mnt/nvme0n1/rgw_datacache/rgw1/:/mnt/nvme0n1/rgw_datacache/rgw1/"
	"-v"
	"/mnt/nvme0n1/rgw_datacache/rgw2/:/mnt/nvme0n1/rgw_datacache/rgw2/"
"

Example for Ceph Object Gateway specification YAML file

[ceph: root@host01 /]# cat rgw-spec.yml
	service_type: rgw
		service_id: rgw.test
		placement:
			hosts:
				host1
				host2
			count-per-host: 2
			extra_container_args:
				"-v"
				"/mnt/nvme0n1/rgw_datacache/rgw1/:/mnt/nvme0n1/rgw_datacache/rgw1/"
				"-v"
				"/mnt/nvme0n1/rgw_datacache/rgw2/:/mnt/nvme0n1/rgw_datacache/rgw2/"

Redeploy the Ceph Object Gateway service by using the ceph orch apply command.
For example,
```
[ceph: root@host01 /]# ceph orch apply -i rgw-spec.yml
```