Pools
The Ceph storage cluster stores data objects in logical partitions called pools. Ceph administrators can create pools for particular types of data, such as for Ceph Block Devices, Ceph Object Gateways, or simply just to separate one group of users from another. Pools play an important role in data durability, performance, and high availability to IBM Storage Ceph.
From the perspective of a Ceph client, the storage cluster is very simple. When a Ceph client reads or writes data using an I/O context, it always connects to a storage pool in the Ceph storage cluster. The client specifies the pool name, a user and a secret key, so the pool appears to act as a logical partition with access controls to its data objects.
A Ceph pool is not only a logical partition for storing object data. A pool plays a critical role in how the Ceph storage cluster distributes and stores data. However, these complex operations are transparent to the Ceph client.
- Pool type
- In early versions of Ceph, a pool simply maintained multiple deep copies of an object. Today, Ceph can maintain multiple copies of an object, or it can use erasure coding to ensure durability. The data durability method is pool-wide, and does not change after creating the pool. The pool type defines the data durability method when creating the pool. Pool types are completely transparent to the client.
- Placement groups
- In an exabyte scale storage cluster, a Ceph pool might store millions of data objects or more. Ceph must handle many types of operations, including data durability via replicas or erasure code chunks, data integrity by scrubbing or CRC checks, replication, rebalancing and recovery. Consequently, managing data on a per-object basis presents a scalability and performance bottleneck. Ceph addresses this bottleneck by sharding a pool into placement groups. The CRUSH algorithm computes the placement group for storing an object and computes the Acting Set of OSDs for the placement group. CRUSH puts each object into a placement group. Then, CRUSH stores each placement group in a set of OSDs. System administrators set the placement group count when creating or modifying a pool.
- CRUSH ruleset
- CRUSH can detect failure domains and performance domains. CRUSH can identify OSDs by storage
media type and organize OSDs hierarchically into nodes, racks, and rows. CRUSH enables Ceph OSDs to
store object copies across failure domains. For example, copies of an object may get stored in
different server rooms, aisles, racks and nodes. If a large part of a cluster fails, such as a rack,
the cluster can still operate in a degraded state until the cluster recovers.Additionally, CRUSH enables clients to write data to particular types of hardware, such as SSDs, hard drives with SSD journals, or hard drives with journals on the same drive as the data. The CRUSH ruleset determines failure domains and performance domains for the pool. Administrators set the CRUSH ruleset when creating a pool.Note: An administrator CANNOT change a pool’s ruleset after creating the pool.
- Durability
- In exabyte scale storage clusters, hardware failure is an expectation and not an exception. When
using data objects to represent larger grained storage interfaces such as a block device, losing one
or more data objects for that larger grained interface can compromise the integrity of the larger
grained storage entity potentially rendering it useless. So data loss is intolerable. Ceph provides
high data durability in two ways:
- Replica pools store multiple deep copies of an object using the CRUSH failure domain to physically separate one data object copy from another. That is, copies get distributed to separate physical hardware. This increases durability during hardware failures.
- Erasure coded pools store each object as
K+Mchunks, whereKrepresents data chunks andMrepresents coding chunks. The sum represents the number of OSDs used to store the object and theMvalue represents the number of OSDs that can fail and still restore data should theMnumber of OSDs fail.