Configuring Crimson

Configure crimson-osd effectively by using pre-built images for deployment. Non-Crimson pools are not supported in a cluster deployed using Crimson image.

Before you begin

Before you begin, make sure that you have the following prerequisites in place:

An IP address for the first Ceph Monitor container, which is also the IP address for the first node in the storage cluster.
Login access to cp.icr.io/cp.
A minimum of 10 GB of available space for /var/lib/containers/.
Root-level access to all nodes.

About this task

Install a new cluster by using the bootstrap option. You cannot upgrade this cluster as it is in the experimental phase. For more information, see Configuring logging and Logging.

CAUTION: Do not use Crimson in a production environment, as it can result in data loss.

Procedure

Use the --image flag to use Crimson build.

cephadm --image cp.icr.io/cp/ibm-ceph/ceph-9-crimson-rhel9:latest bootstrap --mon-ip IP_ADDRESS --allow-fqdn-hostname --initial-dashboard-password PASSWORD --orphan-initial-daemons

Log in to the cephadm shell.
```
cephadm shell
```
Enable Crimson.
Important: Crimson is in a technology preview stage and is not suitable for production use.
```
ceph config set global 'enable_experimental_unrecoverable_data_corrupting_features' crimson
```
Enable the OSD map flag.
Important: This is an irreversible action.
```
ceph osd set-allow-crimson --yes-i-really-mean-it
```
The monitor will not allow crimson-osd to boot without the OSD map flag.
Enable the Crimson parameter for MON to direct the default pools to be created as Crimson pools.
```
ceph config set mon osd_pool_default_crimson true
```
Configure Bluestore options. Since Bluestore is not a native backend for Crimson, deploying Crimson with Bluestore requires configuring specific parameters.

crimson_alien_op_num_threads

Number of CPU threads allocated for Crimson's alienized ObjectStore backend.

Default value: 6

crimson_seastar_num_threads

This option simplifies cluster deployments by avoiding manual setting of CPU mask for each OSD.

Each OSD will utilize crimson_seastar_num_threads number of threads for its operations and the utilized CPU cores will not be exclusive to any individual OSD.

This parameter is not assigned a default value, the user is expected to set this parameter at OSD level depending on their system configuration and cluster requirement before OSD deployment.
Note: It is recommended that the value of crimson_seastar_num_threads multiplied with the number of OSDs on each host, should be less than the number of CPU cores (nproc) on the host

The following demonstrates the calculation of crimson_seastar_num_threads parameter for a 7-node cluster, consisting of 3 OSD hosts, where each OSD host is equipped with 48 CPU cores and 5 spare disks.
- Total number of OSDs that can be deployed on each OSD host: 5
- Number of available CPU cores per node: 48
- Number of threads allocated to Crimson’s alienized ObjectStore (crimson_alien_op_num_threads): 6 (assuming the default value is unchanged)
- The total number of Seastar threads should satisfy the condition:
```
(OSD count * crimson_seastar_num_threads) < nproc
nproc / osd count
```
- Number of threads to be allocated to the Seastar reactor (crimson_seastar_num_threads):
```
(OSD count * crimson_seastar_num_threads) < nproc

crimson_seastar_num_threads < nproc / OSD count  

crimson_seastar_num_threads < 48 / 5

crimson_seastar_num_threads < 9
```
1. Check the default value of the crimson_alien_op_num_threads OSD parameter, by using the config get osd command.
  For example,
```
[ceph: root@host01 /]# ceph config get osd crimson_alien_op_num_threads
6
```
2. Optional: If needed, modify the crimson_alien_op_num_threads OSD parameter by using the ceph config set osd command.
3. Modify the crimson_seastar_num_threads OSD parameter.
  Note: The crimson_alien_op_num_threads OSD parameter can also be modified in a similar manner.
```
ceph config set osd crimson_seastar_num_threads NUMBER_OF_THREADS
```
  For example,
```
[ceph: root@host01 /]# ceph config set osd crimson_seastar_num_threads 8
```
4. Verify the new configuration, by using the ceph config get command.
  For example,
```
[ceph: root@host01 /]# ceph config get osd crimson_seastar_num_threads 8
```
Configure CPU allocation according to the resources available in each host using the crimson_seastar_num_threads option.

Note: It is recommended that the value of crimson_seastar_num_threads multiplied with the number of OSDs on each host, should be less than the number of CPU cores (nproc) on the host.