Scrubbing options

Ceph ensures data integrity by scrubbing placement groups.

The following are the Ceph scrubbing options that you can adjust to increase or decrease scrubbing operations.

You can set these configuration options with the ceph config set global CONFIGURATION_OPTION VALUE command.
mds_max_scrub_ops_in_progress

Description: The maximum number of scrub operations performed in parallel. You can set this value with ceph config set mds_max_scrub_ops_in_progress VALUE command.

Type: integer

Default: 5

osd_max_scrubs

Description: The maximum number of simultaneous scrub operations for a Ceph OSD Daemon.

Type: integer

Default: 1

osd_scrub_begin_hour

Description: The specific hour at which the scrubbing begins. Along with osd_scrub_end_hour, you can define a time window in which the scrubs can happen. Use osd_scrub_begin_hour = 0 and osd_scrub_end_hour = 0 to allow scrubbing the entire day.

Type: integer

Default: 0

Allowed range: [0, 23]

osd_scrub_end_hour

Description: The specific hour at which the scrubbing ends. Along with osd_scrub_begin_hour, you can define a time window, in which the scrubs can happen. Use osd_scrub_begin_hour = 0 and osd_scrub_end_hour = 0 to allow scrubbing for the entire day.

Type: integer

Default: 0

Allowed range: [0, 23]

osd_scrub_begin_week_day

Description: The specific day on which the scrubbing begins. 0 = Sunday, 1 = Monday, etc. Along with osd_scrub_end_week_day, you can define a time window in which scrubs can happen. Use osd_scrub_begin_week_day = 0 and osd_scrub_end_week_day = 0 to allow scrubbing for the entire week.

Type: integer

Default: 0

Allowed range: [0, 6]

osd_scrub_end_week_day

Description: This defines the day on which the scrubbing ends. 0 = Sunday, 1 = Monday, etc. Along with osd_scrub_begin_week_day, they define a time window, in which the scrubs can happen. Use osd_scrub_begin_week_day = 0 and osd_scrub_end_week_day = 0 to allow scrubbing for the entire week.

Type: integer

Default: 0

Allowed range: [0, 6]

osd_scrub_during_recovery

Description: Allow scrub during recovery. Setting this to false disables scheduling new scrub, and deep-scrub, while there is an active recovery. The already running scrubs continue which is useful to reduce load on busy storage clusters.

Type: boolean

Default: false

osd_scrub_extended_sleep

Description: Duration to inject a delay during scrubbing out of scrubbing hours or seconds.

Type: float

Default: 0.0

osd_scrub_backoff_ratio

Description: Backoff ratio for scheduling scrubs. This is the percentage of ticks that do NOT schedule scrubs, 66% means that 1 out of 3 ticks schedules scrubs.

Type: float

Default: 0.66

osd_scrub_load_threshold

Description: The normalized maximum load. Scrubbing does not happen when the system load, as defined by getloadavg()/number of online CPUs, is higher than this defined number.

Type: float

Default: 0.5

osd_scrub_min_interval

Description: The minimal interval in seconds for scrubbing the Ceph OSD daemon when the Ceph storage Cluster load is low.

Type: float

Default: 1 day

osd_scrub_max_interval

Description: The maximum interval in seconds for scrubbing the Ceph OSD daemon irrespective of cluster load.

Type: float

Default: 7 days

osd_scrub_chunk_min

Description: The minimal number of object store chunks to scrub during a single operation. Ceph blocks writes to a single chunk during scrub.

Type: integer

Default: 5

osd_scrub_chunk_max

Description: The maximum number of object store chunks to scrub during a single operation.

Type: integer

Default: 25

osd_scrub_sleep

Description: Time to sleep before scrubbing the next group of chunks. Increasing this value slows down the overall rate of scrubbing, so that client operations are less impacted.

Type: float

Default: 0.0

osd_deep_scrub_interval

Description: The interval for deep scrubbing, fully reading all data. The osd_scrub_load_threshold does not affect this setting.

Type: float

Default: 7 days

osd_debug_deep_scrub_sleep

Description: Inject an expensive sleep during deep scrub IO to make it easier to induce preemption.

Type: float

Default: 0

osd_scrub_interval_randomize_ratio

Description: Add a random delay to osd_scrub_min_interval when scheduling the next scrub job for a placement group. The delay is a random value less than osd_scrub_min_interval * osd_scrub_interval_randomized_ratio. The default setting spreads scrubs throughout the allowed time window of [1, 1.5] * osd_scrub_min_interval.

Type: float

Default: 0.5

osd_deep_scrub_stride

Description: Read size when doing a deep scrub.

Type: size

Default: 512 KB

osd_scrub_auto_repair_num_errors

Description: Auto repair does not occur if more than this many errors are found.

Type: integer

Default: 5

osd_scrub_auto_repair

Description: Setting this to true enables automatic Placement Group (PG) repair when errors are found by scrubs or deep-scrubs. However, if more than osd_scrub_auto_repair_num_errors errors are found, a repair is NOT performed.

Type: boolean

Default: false

osd_scrub_max_preemptions

Description: Set the maximum number of times you need to preempt a deep scrub due to a client operation before blocking client IO to complete the scrub.

Type: integer

Default: 5

osd_deep_scrub_keys

Description: Number of keys to read from an object at a time during deep scrub.

Type: integer

Default: 1024