Limitations of bucket index resharding

Edit online

IMPORTANT: Use the following limitations with caution. There are implications related to your hardware selections, so you should always discuss these requirements with your IBM account team.

  • Maximum number of objects in one bucket before it needs resharding: Use a maximum of 102,400 objects per bucket index shard. To take full advantage of resharding and maximize parallelism, provide a sufficient number of OSDs in the Ceph Object Gateway bucket index pool. This parallelization scales with the number of Ceph Object Gateway instances, and replaces the in-order index shard enumeration with a number sequence. The default locking timeout is extended from 60 seconds to 90 seconds.

  • Maximum number of objects when using sharding: Based on prior testing, the number of bucket index shards currently supported is 65521.

  • You can reshard a bucket three times before the other zones catch-up: Resharding is not recommended until the older generations synchronize. Around four generations of the buckets from previous reshards are supported. Once the limit is reached, dynamic resharding does not reshard the bucket again until at least one of the old log generations are fully trimmed. Using the command radosgw-admin bucket reshard throws the following error:

    Bucket BUCKET_NAME already has too many log generations (4) from previous reshards that peer zones haven't finished syncing.

Resharding is not recommended until the old generations sync, but you can force a reshard with `--yes-i-really-mean-it`.