Managing OSDs

As a storage administrator, you can use the Ceph Orchestrators to manage IBM Storage Ceph OSDs.

Deploy one OSD per physical storage device, using the entire device. This recommendation applies to all supported device types, including HDDs, SSDs, and NVMe drives.
  • Do not deploy multiple OSDs per device for NVMe SSDs.
  • This guidance concerns the main OSD block device, not SSDs used for WAL+DB metadata offload from HDD OSDs.
Note: In some cases, deploying multiple OSDs per device may be appropriate. In the following cases, contact IBM Support for configuration guidance.
  • Ceph Object Gateway workloads with a large number of very small S3 objects or many object versions.
  • Very large NVMe SSDs (30 TB or larger).
Check the capacity of a cluster regularly to see whether it is reaching the upper end of its storage capacity. As a storage cluster reaches its near full ratio, add one or more OSDs to expand the storage cluster’s capacity. If the node has multiple storage drives, you might also need to remove one of the ceph-osd daemons for that drive.
Important:
  • Ensure that when you remove an OSD, the storage cluster is not at its near full ratio.
  • Do not allow a storage cluster to reach the full ratio before adding an OSD. OSD failures that occur after the storage cluster reaches the near full ratio can cause the storage cluster to exceed the full ratio. Ceph blocks write access to protect the data until you resolve the storage capacity issues. When removing an OSD, be sure to consider the impact on the full ratio first.

Configure Ceph OSDs and their supporting hardware in a similar manner as a storage strategy for any pools that will use the OSDs. Ceph prefers uniform hardware across pools for a consistent performance profile. For best performance, consider a CRUSH hierarchy with drives of the same type or size.

In cases where you add drives of dissimilar size, adjust their weights accordingly. When you add the OSD to the CRUSH map, consider the weight for the new OSD. Hard disk drive capacity grows approximately 40% per year, so newer OSD nodes might have larger hard disk drives than older nodes in the storage cluster, that is, they might have a greater weight.