Configuring a CRUSH map for stretch mode

Use this information to configure a CRUSH map for stretch mode. In this step, we modify few configs on the cluster, add new crush rule and enable stretch mode on the cluster.

Before you begin

Before you begin, make sure that you have the following prerequisites in place:

Root-level access to the nodes.
The CRUSH location is set to the hosts.

Procedure

Create a CRUSH rule that makes use of this OSD crush topology by installing the ceph-base RPM package in order to use the crushtool command.
```
dnf -y install ceph-base
```

Get the compiled CRUSH map from the cluster.

ceph osd getcrushmap > /etc/ceph/crushmap.bin

Decompile the CRUSH map and convert it to a text file to edit it.
```
crushtool -d /etc/ceph/crushmap.bin -o /etc/ceph/crushmap.txt
```

Add the following rule to the CRUSH map by editing the /etc/ceph/crushmap.txt at the end of the file.

This rule distributes reads and writes evenly across the data center.

rule stretch_rule {
        id 1 
        type replicated
        step take default
        step choose firstn 0 type datacenter
        step chooseleaf firstn 2 type host
        step emit
 }

Optionally have the cluster with a read/write affinity towards data center 1.

rule stretch_rule {
         id 1
         type replicated
         step take DC1
         step chooseleaf firstn 2 type host
         step emit
         step take DC2
         step chooseleaf firstn 2 type host
         step emit
 }

The CRUSH rule declared contains the following information:
     Rule name
          Description: A unique name for identifying the rule.
          Value: stretch_rule
     id
          Description: A unique whole number for identifying the rule.
          Value: 1
     type
          Description: Describes a rule for either a storage drive replicated or erasure-coded.
          Value: replicated
     step take default
          Description: Takes the root bucket called default, and begins iterating down the tree.
     step take DC1
          Description: Takes the bucket called DC1, and begins iterating down the tree.
     step choose firstn 0 type datacenter
          Description: Selects the datacenter bucket, and goes into its subtrees.
     step chooseleaf firstn 2 type host
          Description: Selects the number of buckets of the given type. In this case, it is two different hosts located in the datacenter it entered at the previous level.
     step emit
          Description: Outputs the current value and empties the stack. Typically used at the end of a rule, but may also be used to pick from different trees in the same rule.

Compile the new CRUSH map from /etc/ceph/crushmap.txt and convert it to a binary file /etc/ceph/crushmap2.bin.

crushtool -c /path/to/crushmap.txt -o /path/to/crushmap2.bin

For example,

[ceph: root@host01 /]# crushtool -c /etc/ceph/crushmap.txt -o /etc/ceph/crushmap2.bin

Inject the newly created CRUSH map back into the cluster.
```
ceph osd setcrushmap -i /path/to/compiled_crushmap
```
For example,
```
ceph osd setcrushmap -i /etc/ceph/crushmap2.bin
```
Note: The number 17 is a counter and increases (18,19, and so on) depending on the changes that are made to the CRUSH map.

What to do next

Verify that the newly created stretch_rule available for use.

ceph osd crush rule ls

For example,

[ceph: root@host01 /]# ceph osd crush rule ls

replicated_rule
stretch_rule