Pools, placement groups and CRUSH

When you create pools and set the number of placement groups for the pool, Ceph uses default values when you do not specifically override the defaults.

You can set these values when running pool commands.

By default, Ceph makes three replicas of objects. If you want to set four copies of an object as the default value, a primary copy and three replica copies, reset the default values as shown in osd_pool_default_size. If you want to allow Ceph to write a lesser number of copies in a degraded state, set osd_pool_default_min_size to a number less than the osd_pool_default_size value.

[ceph: root@host01 /]# ceph config set global osd_pool_default_size 4  # Write an object four times.
[ceph: root@host01 /]# ceph config set global osd_pool_default_min_size 1  # Allow writing one copy in a degraded state.

Ensure you have a realistic number of placement groups. IBM recommends approximately 100 per OSD. Total number of OSDs multiplied by 100 divided by the number of replicas, that is, osd_pool_default_size. For 10 OSDs and osd_pool_default_size = 4, we would recommend approximately (100*10)/4=250.

Example

[ceph: root@host01 /]# ceph config set global osd_pool_default_pg_num 250
[ceph: root@host01 /]# ceph config set global osd_pool_default_pgp_num 250