Network configuration options

Common options

These are the common network configuration options for Ceph.

public_network: Description: The IP address and netmask of the public (front-side) network (for example, 192.168.0.0/24). Set in [global]. You can specify comma-delimited subnets.; Type: <ip-address>/<netmask> [, <ip-address>/<netmask>]; Required: No; Default: N/A
public_addr: Description: The IP address for the public (front-side) network. Set for each daemon.; Type: IP Address; Required: No; Default: N/A
cluster_network: Description: The IP address and netmask of the cluster network (for example, 10.0.0.0/24). Set in [global]. You can specify comma-delimited subnets.; Type: <ip-address>/<netmask> [, <ip-address>/<netmask>]; Required: No; Default: N/A
cluster_addr: Description: The IP address for the cluster network. Set for each daemon.; Type: Address; Required: No; Default: N/A
ms_type: Description: The messenger type for the network transport layer. IBM supports the simple and the async messenger type using posix semantics.; Type: String.; Required: No.; Default: async+posix
ms_public_type: Description: The messenger type for the network transport layer of the public network. It operates identically to ms_type, but is applicable only to the public or front-side network. This setting enables Ceph to use a different messenger type for the public or front-side and cluster or back-side networks.; Type: String.; Required: No.; Default: None.
ms_cluster_type: Description: The messenger type for the network transport layer of the cluster network. It operates identically to ms_type, but is applicable only to the cluster or back-side network. This setting enables Ceph to use a different messenger type for the public or front-side and cluster or back-side networks.; Type: String.; Required: No.; Default: None.

Host options

You must declare at least one Ceph Monitor in the Ceph configuration file, with a mon addr setting under each declared monitor. Ceph expects a host setting under each declared monitor, metadata server and OSD in the Ceph configuration file.

Important: Do not use localhost. Use the short name of the node, not the fully-qualified domain name (FQDN). Do not specify any value for host when using a third party deployment system that retrieves the node name for you.

mon_addr: Description: A list of <hostname>:<port> entries that clients can use to connect to a Ceph monitor. If not set, Ceph searches [mon.*] sections.; Type: String; Required: No; Default: N/A
host: Description: The host name. Use this setting for specific daemon instances (for example, [osd.0]).; Type: String; Required: Yes, for daemon instances.; Default: localhost

TCP options

Ceph disables TCP buffering by default.

ms_tcp_nodelay: Description: Ceph enables ms_tcp_nodelay so that each request is sent immediately (no buffering). Disabling Nagle’s algorithm increases network traffic, which can introduce congestion. If you experience large numbers of small packets, you may try disabling ms_tcp_nodelay, but be aware that disabling it will generally increase latency.; Type: Boolean; Required: No; Default: true
ms_tcp_rcvbuf: Description: The size of the socket buffer on the receiving end of a network connection. Disabled by default.; Type: 32-bit Integer; Required: No; Default: 0

Bind options

The bind options configure the default port ranges for the Ceph OSD daemons. The default range is 6800:7100. You can also enable Ceph daemons to bind to IPv6 addresses.

Important: Verify that the firewall configuration allows you to use the configured port range.

ms_bind_port_min: Description: The minimum port number to which an OSD daemon will bind.; Type: 32-bit Integer; Default: 6800; Required: No
ms_bind_ipv6: Description: Enables Ceph daemons to bind to IPv6 addresses.; Type: Boolean; Default: false; Required: No

Asynchronous messenger options

These Ceph messenger options configure the behavior of AsyncMessenger.

ms_async_op_threads: Description: Initial number of worker threads used by each AsyncMessenger instance. This configuration setting SHOULD equal the number of replicas or erasure code chunks, but it may be set lower if the CPU core count is low or the number of OSDs on a single server is high.; Type: 64-bit Unsigned Integer; Required: No; Default: 3