Deploying the Ceph Object Gateway using the service specification

Deploy the Ceph Object Gateway using the service specification with either the default or the custom realms, zones, and zone groups.

Prerequisites

A running IBM Storage Ceph cluster.
Root-level access to the bootstrapped host.
Hosts are added to the cluster.
All manager, monitor, and OSD daemons are deployed.

Procedure

As a root user, create a specification file.
```
touch rgw_spec.yaml
```
For example,
```
[root@host01 ~]# touch rgw_spec.yaml
```
Configure S3 requests to wait for the duration defined in the rgw_exit_timeout_secs parameter for all outstanding requests to complete by setting rgw_graceful_stop to true during Ceph Object gateway shutdown/restart.
```
ceph config set client.rgw rgw_graceful_stop true

ceph config set client.rgw rgw_exit_timeout_secs 120
```
Note: In containerized deployments, an additional extra_container_args configuration of --stop-timeout=120 (or the value of rgw_exit_timeout_secs configuration, if not default) is also necessary in order for it to work as expected with ceph orch stop/restart commands.
For example,
```
[root@host1 ~]$ cat rgw_spec.yaml 
service_type: rgw
service_id: foo
placement:
  count_per_host: 1
  hosts:
    - rgw_node
spec:
  rgw_frontend_port: 8081
extra_container_args:
  - "--stop-timeout=120"
```

Edit the rgw_spec.yaml file to include the following details for the default realm, zone, and zone group.

service_type: rgw
service_id: REALM_NAME.ZONE_NAME
placement:
  hosts:
  - HOST_NAME_1
  - HOST_NAME_2
  count_per_host: NUMBER_OF_DAEMONS
spec:
  rgw_realm: REALM_NAME
  rgw_zone: ZONE_NAME
  rgw_zonegroup: ZONE_GROUP_NAME
  rgw_frontend_port: FRONT_END_PORT
networks:
  -  NETWORK_CIDR # Ceph Object Gateway service binds to a specific network

NUMBER_OF_DAEMONS controls the number of Ceph Object Gateways deployed on each host. To achieve the highest performance without incurring an additional cost, set this value to 2.

For example,

service_type: rgw
service_id: default
placement:
  hosts:
  - host01
  - host02
  - host03
  count_per_host: 2
spec:
  rgw_realm: default
  rgw_zone: default
  rgw_zonegroup: default
  rgw_frontend_port: 1234
networks:
  - 192.169.142.0/24

Optional: For custom realm, zone, and zone group, create the resources and then create the rgw_spec.yaml file.

Create the custom realm, zone, and zone group.

For example,

[root@host01 ~]# radosgw-admin realm create --rgw-realm=test_realm --default
[root@host01 ~]# radosgw-admin zonegroup create --rgw-zonegroup=test_zonegroup --default
[root@host01 ~]# radosgw-admin zone create --rgw-zonegroup=test_zonegroup --rgw-zone=test_zone --default
[root@host01 ~]# radosgw-admin period update --rgw-realm=test_realm --commit

Create the rgw_spec.yaml file with the following details.

For example,

service_type: rgw
service_id: test_realm.test_zone
placement:
  hosts:
  - host01
  - host02
  - host03
  count_per_host: 2
spec:
  rgw_realm: test_realm
  rgw_zone: test_zone
  rgw_zonegroup: test_zonegroup
  rgw_frontend_port: 1234
networks:
  - 192.169.142.0/24

Optional: Deploy HA Proxy concentrator for local RGW load balancing. To load balance requests across RGW daemons on the same host, configure the RGW service with a local HA Proxy concentrator.
Note: HA Proxy concentrator does not currently support SSL.
Note: Use this only when count_per_host > 1.
For example,
```
service_type: rgw
service_id: foo
service_name: rgw.foo
placement:
  count_per_host: 2
  hosts:
    - host1
    - host2
spec:
  concentrator: haproxy
  concentrator_frontend_port: 8080
  concentrator_monitor_port: 1967
  concentrator_monitor_user: admin
 
```
Mount the rgw_spec.yaml file under a directory in the container.
For example,
```
[root@host01 ~]# cephadm shell --mount rgw_spec.yaml:/var/lib/ceph/radosgw/rgw_spec.yaml
```
Note: Every time you exit the shell, you have to mount the file in the container before deploying the daemon.

Deploy the Ceph Object Gateway using the service specification.

ceph orch apply -i FILE_NAME.yaml

For example,

[ceph: root@host01 /]# ceph orch apply -i /var/lib/ceph/radosgw/rgw_spec.yaml

Note: In containerized deployments, an additional extra_container_args configuration of --stop-timeout=120 (or the value of rgw_exit_timeout_secs configuration, if not default) is also necessary.

For example,

[root@host1 ~]$ cat rgw_spec.yaml 
service_type: rgw
service_id: foo
placement:
  count_per_host: 1
  hosts:
    - rgw_node
spec:
  rgw_frontend_port: 8081
extra_container_args:
  - "--stop-timeout=120"

Verification

List the service.
```
ceph orch ls
```

List the hosts, daemons, and processes.

ceph orch ps --daemon_type=DAEMON_NAME

For example,

[ceph: root@host01 /]# ceph orch ps --daemon_type=rgw