Adding D3N cache directory

To enable D3N cache on Ceph Object Gateway, you need to also include the D3N cache directory in the podman unit.run file.

Before you begin

A running IBM Storage Ceph cluster.
Ceph Object Gateway installed.
Root-level access to the admin node.
A fast NVMe drive in each Ceph Object Gateway node to serve as the local cache storage.

Procedure

Create a mount point for the NVMe drive.

mkfs.ext4 NVME_DRIVE_PATH

For example,

[ceph: root@host01 /]# mkfs.ext4 /dev/nvme0n1
mount /dev/nvme0n1 /mnt/nvme0n1/

Create a cache directory path.

mkdir NVME_MOUNT_PATH/CACHE_DIRECTORY_NAME

For example,

[ceph: root@host01 /]# mkdir /mnt/nvme0n1/rgw_datacache

Provide a+rwx permission to NVME_MOUNT_PATH and to the rgw_d3n_l1_datacache_persistent_path.

chmod a+rwx NVME_MOUNT_PATH ; chmod a+rwx rgw_d3n_l1_datacache_persistent_path

For example,

[ceph: root@host01 /]# chmod a+rwx /mnt/nvme0n1 ; chmod a+rwx /mnt/nvme0n1/rgw_datacache/

Create or modify the Ceph Object Gateway specification file with extra_container_args to add rgw_d3n_l1_datacache_persistent_path into podman unit.run file.

"extra_container_args:
          "-v"
          "rgw_d3n_l1_datacache_persistent_path:rgw_d3n_l1_datacache_persistent_path"
			"

For example,

[ceph: root@host01 /]# cat rgw-spec.yml
	service_type: rgw
	service_id: rgw.test
	placement:
		hosts:
		    host1
		    host2
		extra_container_args:
		    "-v"
		    "/mnt/nvme0n1/rgw_datacache/:/mnt/nvme0n1/rgw_datacache/"

If there are multiple instances of Ceph Object Gateway in a single host, then a separate rgw_d3n_l1_datacache_persistent_path must be created for each instance and add each path in extra_container_args.

For two instances of Ceph Object Gateway in each host, create two separate instances of cache-directory under rgw_d3n_l1_datacache_persistent_path, /mnt/nvme0n1/rgw_datacache/rgw1 and /mnt/nvme0n1/rgw_datacache/rgw2.

Example for extra_container_args in a Ceph Object Gateway specification file

"extra_container_args:
	"-v"
	"/mnt/nvme0n1/rgw_datacache/rgw1/:/mnt/nvme0n1/rgw_datacache/rgw1/"
	"-v"
	"/mnt/nvme0n1/rgw_datacache/rgw2/:/mnt/nvme0n1/rgw_datacache/rgw2/"
"

Example for Ceph Object Gateway specification YAML file

[ceph: root@host01 /]# cat rgw-spec.yml
	service_type: rgw
		service_id: rgw.test
		placement:
			hosts:
				host1
				host2
			count-per-host: 2
			extra_container_args:
				"-v"
				"/mnt/nvme0n1/rgw_datacache/rgw1/:/mnt/nvme0n1/rgw_datacache/rgw1/"
				"-v"
				"/mnt/nvme0n1/rgw_datacache/rgw2/:/mnt/nvme0n1/rgw_datacache/rgw2/"

Redeploy the Ceph Object Gateway service by using the ceph orch apply command.
For example,
```
[ceph: root@host01 /]# ceph orch apply -i rgw-spec.yml
```