Ceph OSD metrics

Write back throttle metrics

Table 1 describes the write back throttle metrics for the WBThrottle collection.

Table 1. Write back throttle metrics table
Metric Name	Bit Field Value	Short Description
`bytes_dirtied`	2	Dirty data
`bytes_wb`	2	Written data
`ios_dirtied`	2	Dirty operations
`ios_wb`	2	Written operations
`inodes_dirtied`	2	Entries waiting for write
`inodes_wb`	2	Written entries

Level database metrics

Table 2 describes the level database metrics for the leveldb collection.

Table 2. Level database metrics table
Collection Name	Metric Name	Bit Field Value	Short Description
`leveldb`	`leveldb_get`	10	Gets
	`leveldb_transaction`	10	Transactions
	`leveldb_compact`	10	Compactions
	`leveldb_compact_range`	10	Compactions by range
	`leveldb_compact_queue_merge`	10	Merging of ranges in compaction queue
	`leveldb_compact_queue_len`	2	Length of compaction queue

Objecter metrics

Table 3 describes the objecter metrics for the objecter collection.

Table 3. Objecter metrics table
Metric Name	Bit Field Value	Short Description
`op_active`	2	Active operations
`op_laggy`	2	Laggy operations
`op_send`	10	Sent operations
`op_send_bytes`	10	Sent data
`op_resend`	10	Resent operations
`op_ack`	10	Commit callbacks
`op_commit`	10	Operation commits
`op`	10	Operation
`op_r`	10	Read operations
`op_w`	10	Write operations
`op_rmw`	10	Read-modify-write operations
`op_pg`	10	PG operation
`osdop_stat`	10	Stat operations
`osdop_create`	10	Create object operations
`osdop_read`	10	Read operations
`osdop_write`	10	Write operations
`osdop_writefull`	10	Write full object operations
`osdop_append`	10	Append operation
`osdop_zero`	10	Set object to zero operations
`osdop_truncate`	10	Truncate object operations
`osdop_delete`	10	Delete object operations
`osdop_mapext`	10	Map extent operations
`osdop_sparse_read`	10	Sparse read operations
`osdop_clonerange`	10	Clone range operations
`osdop_getxattr`	10	Get xattr operations
`osdop_setxattr`	10	Set xattr operations
`osdop_cmpxattr`	10	Xattr comparison operations
`osdop_rmxattr`	10	Remove xattr operations
`osdop_resetxattrs`	10	Reset xattr operations
`osdop_tmap_up`	10	TMAP update operations
`osdop_tmap_put`	10	TMAP put operations
`osdop_tmap_get`	10	TMAP get operations
`osdop_call`	10	Call (run) operations
`osdop_watch`	10	Watch by object operations
`osdop_notify`	10	Notify about object operations
`osdop_src_cmpxattr`	10	Extended attribute comparison in multi operations
`osdop_other`	10	Other operations
`linger_active`	2	Active lingering operations
`linger_send`	10	Sent lingering operations
`linger_resend`	10	Resent lingering operations
`linger_ping`	10	Sent pings to lingering operations
`poolop_active`	2	Active pool operations
`poolop_send`	10	Sent pool operations
`poolop_resend`	10	Resent pool operations
`poolstat_active`	2	Active get pool stat operations
`poolstat_send`	10	Pool stat operations sent
`poolstat_resend`	10	Resent pool stats
`statfs_active`	2	Statfs operations
`statfs_send`	10	Sent FS stats
`statfs_resend`	10	Resent FS stats
`command_active`	2	Active commands
`command_send`	10	Sent commands
`command_resend`	10	Resent commands
`map_epoch`	2	OSD map epoch
`map_full`	10	Full OSD maps received
`map_inc`	10	Incremental OSD maps received
`osd_sessions`	2	Open sessions
`osd_session_open`	10	Sessions opened
`osd_session_close`	10	Sessions closed
`osd_laggy`	2	Laggy OSD sessions

Read and write operations metrics

Table 4 describes the read and write operations metrics for the osd collection.

Table 4. Read and write operations metrics table
Metric Name	Bit Field Value	Short Description
`op_wip`	2	Replication operations currently being processed (primary)
`op_in_bytes`	10	Client operations total write size
`op_out_bytes`	10	Client operations total read size
`op_latency`	5	Latency of client operations (including queue time)
`op_process_latency`	5	Latency of client operations (excluding queue time)
`op_r`	10	Client read operations
`op_r_out_bytes`	10	Client data read
`op_r_latency`	5	Latency of read operation (including queue time)
`op_r_process_latency`	5	Latency of read operation (excluding queue time)
`op_w`	10	Client write operations
`op_w_in_bytes`	10	Client data written
`op_w_rlat`	5	Client write operation readable/applied latency
`op_w_latency`	5	Latency of write operation (including queue time)
`op_w_process_latency`	5	Latency of write operation (excluding queue time)
`op_rw`	10	Client read-modify-write operations
`op_rw_in_bytes`	10	Client read-modify-write operations write in
`op_rw_out_bytes`	10	Client read-modify-write operations read out
`op_rw_rlat`	5	Client read-modify-write operation readable/applied latency
`op_rw_latency`	5	Latency of read-modify-write operation (including queue time)
`op_rw_process_latency`	5	Latency of read-modify-write operation (excluding queue time)
`subop`	10	Sub operations
`subop_in_bytes`	10	Sub operations total size
`subop_latency`	5	Sub operations latency
`subop_w`	10	Replicated writes
`subop_w_in_bytes`	10	Replicated written data size
`subop_w_latency`	5	Replicated writes latency
`subop_pull`	10	Sub operations pull requests
`subop_pull_latency`	5	Sub operations pull latency
`subop_push`	10	Sub operations push messages
`subop_push_in_bytes`	10	Sub operations pushed size
`subop_push_latency`	5	Sub operations push latency
`pull`	10	Pull requests sent
`push`	10	Push messages sent
`push_out_bytes`	10	Pushed size
`push_in`	10	Inbound push messages
`push_in_bytes`	10	Inbound pushed size
`recovery_ops`	10	Started recovery operations
`loadavg`	2	CPU load
`buffer_bytes`	2	Total allocated buffer size
`numpg`	2	Placement groups
`numpg_primary`	2	Placement groups for which this osd is primary
`numpg_replica`	2	Placement groups for which this osd is replica
`numpg_stray`	2	Placement groups ready to be deleted from this osd
`heartbeat_to_peers`	2	Heartbeat (ping) peers we send to
`heartbeat_from_peers`	2	Heartbeat (ping) peers we recv from
`map_messages`	10	OSD map messages
`map_message_epochs`	10	OSD map epochs
`map_message_epoch_dups`	10	OSD map duplicates
`stat_bytes`	2	OSD size
`stat_bytes_used`	2	Used space
`stat_bytes_avail`	2	Available space
`copyfrom`	10	RADOS copy-from operations
`tier_promote`	10	Tier promotions
`tier_flush`	10	Tier flushes
`tier_flush_fail`	10	Failed tier flushes
`tier_try_flush`	10	Tier flush attempts
`tier_try_flush_fail`	10	Failed tier flush attempts
`tier_evict`	10	Tier evictions
`tier_whiteout`	10	Tier whiteouts
`tier_dirty`	10	Dirty tier flag set
`tier_clean`	10	Dirty tier flag cleaned
`tier_delay`	10	Tier delays (agent waiting)
`tier_proxy_read`	10	Tier proxy reads
`agent_wake`	10	Tiering agent wake up
`agent_skip`	10	Objects skipped by agent
`agent_flush`	10	Tiering agent flushes
`agent_evict`	10	Tiering agent evictions
`object_ctx_cache_hit`	10	Object context cache hits
`object_ctx_cache_total`	10	Object context cache lookups
`ceph_cluster_osd_blocklist_count`	2	Number of clients blocklisted

Recovery state metrics

Table 5 describes the recovery state metrics for the recoverystate_perf collection.

Table 5. Recovery state metrics table
Metric Name	Bit Field Value	Short Description
`initial_latency`	5	Initial recovery state latency
`started_latency`	5	Started recovery state latency
`reset_latency`	5	Reset recovery state latency
`start_latency`	5	Start recovery state latency
`primary_latency`	5	Primary recovery state latency
`peering_latency`	5	Peering recovery state latency
`backfilling_latency`	5	Backfilling recovery state latency
`waitremotebackfillreserved_latency`	5	Wait remote backfill reserved recovery state latency
`waitlocalbackfillreserved_latency`	5	Wait local backfill reserved recovery state latency
`notbackfilling_latency`	5	Notbackfilling recovery state latency
`repnotrecovering_latency`	5	Repnotrecovering recovery state latency
`repwaitrecoveryreserved_latency`	5	Rep wait recovery reserved recovery state latency
`repwaitbackfillreserved_latency`	5	Rep wait backfill reserved recovery state latency
`RepRecovering_latency`	5	RepRecovering recovery state latency
`activating_latency`	5	Activating recovery state latency
`waitlocalrecoveryreserved_latency`	5	Wait local recovery reserved recovery state latency
`waitremoterecoveryreserved_latency`	5	Wait remote recovery reserved recovery state latency
`recovering_latency`	5	Recovering recovery state latency
`recovered_latency`	5	Recovered recovery state latency
`clean_latency`	5	Clean recovery state latency
`active_latency`	5	Active recovery state latency
`replicaactive_latency`	5	Replicaactive recovery state latency
`stray_latency`	5	Stray recovery state latency
`getinfo_latency`	5	Getinfo recovery state latency
`getlog_latency`	5	Getlog recovery state latency
`waitactingchange_latency`	5	Waitactingchange recovery state latency
`incomplete_latency`	5	Incomplete recovery state latency
`getmissing_latency`	5	Get missing recovery state latency
`waitupthru_latency`	5	Waitupthru recovery state latency

OSD throttle metrics

Table 6 describes the OSD throttle metrics for the throttle-* collection.

Table 6. OSD throttle metrics table
Metric Name	Bit Field Value	Short Description
`val`	10	Currently available throttle
`max`	10	Max value for throttle
`get`	10	Gets
`get_sum`	10	Got data
`get_or_fail_fail`	10	Get blocked during get_or_fail
`get_or_fail_success`	10	Successful get during get_or_fail
`take`	10	Takes
`take_sum`	10	Taken data
`put`	10	Puts
`put_sum`	10	Put data
`wait`	5	Waiting latency