Configuring a private network

Edit online

Network configuration settings are not required. Ceph assumes a public network with all hosts operating on it, unless you specifically configure a cluster network, also known as a private network.

If you create a cluster network, OSDs routes heartbeat, object replication, and recovery traffic over the cluster network. This can improve performance, compared to using a single network.
Important: For added security, the cluster network should not be reachable from the public network or the Internet.

To assign a cluster network, use the --cluster-network option with the cephadm bootstrap command. The cluster network that you specify must define a subnet in CIDR notation (for example, 10.90.90.0/24 or fe80::/64).

You can also configure the cluster_network after boostrap.

Prerequisites

  • Access to the Ceph software repository.

  • Root-level access to all nodes in the storage cluster.

Procedure

  1. Run the cephadm bootstrap command from the initial node that you want to use as the Monitor node in the storage cluster. Include the --cluster-network option in the command.

    Syntax 

     cephadm bootstrap --mon-ip IP-ADDRESS --registry-url registry.redhat.io --registry-username USER_NAME --registry-password PASSWORD --cluster-network NETWORK-IP-ADDRESS

    Example 

     [root@host01 ~]# cephadm bootstrap --mon-ip 10.10.128.68 --registry-url registry.redhat.io --registry-username myuser1 --registry-password mypassword1 --cluster-network 10.10.0.0/24

  2. To configure the cluster_network after bootstrap, run the config set command and redeploy the daemons:

    a. Log in to the cephadm shell:

    Example 

             [root@host01 ~]# cephadm shell

    b. Configure the cluster network with the subnet:

    Syntax 

         ceph config set global cluster_network IP_ADDRESS_WITH_SUBNET

    Example 

     [ceph: root@host01 /]# ceph config set global cluster_network 10.10.0.0/24

    c. Get the list of services in the storage cluster:

    Example 

     [ceph: root@host01 /]# ceph orch ls

    d. Restart the daemons. Ceph daemons bind dynamically, so you do not have to restart the entire cluster at once if you change the network configuration for a specific daemon.

    Example 

     [ceph: root@host01 /]# ceph orch restart mon

    e. Optional: If you want to restart the cluster, on the admin node as a root user, run systemctl command:

    Syntax 

     systemctl restart ceph-FSID_OF_CLUSTER.target

    Example 

     [root@host01 ~]# systemctl restart ceph-1ca9f6a8-d036-11ec-8263-fa163ee967ad.target

Reference