Increasing the storage capacity of a GlusterFS cluster

Increase the storage capacity of your GlusterFS cluster.

You can increase the storage capacity of your GlusterFS cluster by adding a device on an existing storage node or by adding a storage node to the GlusterFS cluster.

• Adding a device on a GlusterFS storage node
• Adding a storage node to the GlusterFS cluster

Before you begin, set up the kubectl CLI. See Accessing your cluster from the kubectl CLI.

Note: Whenever you use -- heketi-cli in a kubectl command, you must provide the --user admin and --secret <admin_password> parameters for Heketi pod authentication. <admin_password> is the password that is used to create the Heketi authentication secret during GlusterFS installation.

Adding a device on a GlusterFS storage node

You can add a device on a storage node that exists in your GlusterFS cluster. The device must meet the GlusterFS device requirements. See the Hardware requirements.

After you add the device, to make it a part of the Heketi topology, first retrieve the required information, then run the command for updating the topology.

Retrieving the required information

You need the ID of the node on which you are adding a device. To get the node ID, run the following commands:

1. Get the Heketi pod name.

   kubectl -n kube-system get pod -l glusterfs=heketi-pod
   

   Following is a sample output:

   NAME                      READY     STATUS    RESTARTS   AGE
   heketi-7b69bb4d48-g4kx9   1/1       Running   0          1h
   

2. Get the cluster ID.

   kubectl -n kube-system exec <Heketi pod name> -- heketi-cli --user admin --secret <admin_password> cluster list
   

   Following is a sample command and output:

   kubectl -n kube-system exec heketi-7b69bb4d48-g4kx9 -- heketi-cli --user admin --secret admin cluster list
   
   Clusters:
   Id:2a47436e012604919dae5fbb4fc3899c [file][block]
   #
   

3. Get the node ID:

   kubectl -n kube-system exec <Heketi pod name> -- heketi-cli --user admin --secret <admin_password> cluster info <cluster ID>
   

   The command returns a list of nodes.

   Following is a sample command and output:

   kubectl -n kube-system exec heketi-7b69bb4d48-g4kx9 -- heketi-cli --user admin --secret admin cluster info 2a47436e012604919dae5fbb4fc3899c
   
   Cluster id: 2a47436e012604919dae5fbb4fc3899c
   Nodes:
   6c4c66317f5650e7b61057fede8dbe25
   8164c019b2e7c67b224709a97d05d0e9
   de3d0a9a21868a09f92fb50e166cceda
   Volumes:
   896044a59b18316790ff5554455a7a9e
   b971b2bb34703143e094f9b5241e4ddf
   f3f93e7eddd48e95779361ff35bd2222
   Block: true
   
   File: true
   

4. Identify the node that has the new device. You can do so by the node IP address. To retrieve the node IP address, run the following command for all the nodes. Make a note of the node ID of the storage node that has the new device.

   kubectl -n kube-system exec <Heketi pod name> -- heketi-cli --user admin --secret <admin_password> node info <node ID>
   

   Note: Get the symlink of the new device. For more information about how to get the symlink, see Symlinks.

   Following is a sample command and output:

   kubectl -n kube-system exec heketi-7b69bb4d48-g4kx9 -- heketi-cli --user admin --secret admin node info de3d0a9a21868a09f92fb50e166cceda
   
   Node Id: de3d0a9a21868a09f92fb50e166cceda
   State: online
   Cluster Id: 2a47436e012604919dae5fbb4fc3899c
   Zone: 1
   Management Hostname: 192.168.0.96
   Storage Hostname: 192.168.0.96
   Devices:
   Id:4c6ba3295f71a78515d6bd5d5aad58ae   Name:/dev/disk/by-path/pci-0000_00_10_0    State:online    Size (GiB):24      Used     (GiB):2       Free (GiB):22
   #
   

Updating the topology

Update the Heketi topology with the information about the new device. Run the following command:

kubectl -n kube-system exec <Heketi pod name> -- heketi-cli --user admin --secret <admin_password> device add --name=<device symlink> --node=<Node ID>

Following is a sample command and output:

kubectl -n kube-system exec heketi-7b69bb4d48-g4kx9 -- heketi-cli --user admin --secret admin device add --name=/dev/disk/by-path/pci-0000_00_11_0 --node=de3d0a9a21868a09f92fb50e166cceda
Device added successfully
#

Adding a storage node to the GlusterFS cluster

You can add a storage node to your GlusterFS cluster. The device on the new storage node must meet the GlusterFS device requirements. See the Hardware requirements.

Before you proceed, complete these steps:

• Add the new node to the config.yaml file in the /<installation_directory>/cluster folder. See Configuring GlusterFS during IBM Cloud Private installation.
• Add the new node to the custom host group that you created with GlusterFS storage nodes. For more information, see Adding a host group.

Complete the following steps by using the kubectl CLI:

1. Get the GlusterFS pod name.

   kubectl -n kube-system get pod -owide -l glusterfs=pod
   

   Following is a sample command and output:

   kubectl -n kube-system get pod -owide -l glusterfs=pod
   
   NAME              READY     STATUS              RESTARTS   AGE       IP              NODE
   glusterfs-4srll   1/1       Running             1          3h        192.168.0.96    192.168.0.96
   glusterfs-5zp55   0/1       ContainerCreating   0          7s        192.168.0.139   192.168.0.139
   glusterfs-j2t6d   1/1       Running             0          3h        192.168.0.56    192.168.0.56
   glusterfs-q8mcm   1/1       Running             0          3h        192.168.0.184   192.168.0.184
   #
   

2. Run the peer probe command from any of the existing GlusterFS pods.

   kubectl -n kube-system exec <GlusterFS pod name> -- gluster peer probe <IP address or host name of the new node>
   

   Following is a sample command and output:

   kubectl -n kube-system exec glusterfs-4srll -- gluster peer probe 192.168.0.139
   peer probe: success.
   #
   

3. Get the Heketi pod name.

   kubectl -n kube-system get pod -l glusterfs=heketi-pod
   

   Following is a sample output:

   NAME                      READY     STATUS    RESTARTS   AGE
   heketi-7b69bb4d48-g4kx9   1/1       Running   0          1h
   

4. Get the cluster ID.

   kubectl -n kube-system exec <Heketi pod name> -- heketi-cli --user admin --secret <admin_password> cluster list
   

   Following is a sample command and output:

   kubectl -n kube-system exec heketi-7b69bb4d48-g4kx9 -- heketi-cli --user admin --secret admin cluster list
   
   Clusters:
   Id:2a47436e012604919dae5fbb4fc3899c [file][block]
   #
   

5. Add the new node to the GlusterFS cluster.

   kubectl -n kube-system exec <Heketi pod name> -- heketi-cli --user admin --secret <admin_password> node add --zone=1 --cluster=<cluster ID> --management-host-name=<IP address or host name of the new node> --storage-host-name=<IP address of the new node>
   

   Following is a sample command and output:

   kubectl -n kube-system exec heketi-7b69bb4d48-g4kx9 -- heketi-cli --user admin --secret admin node add --zone=1 --cluster=2a47436e012604919dae5fbb4fc3899c --management-host-name=192.168.0.139 --storage-host-name=192.168.0.139
   
   Node information:
   Id: 5df753379b9629eda0c5eb71687ee2ea
   State: online
   Cluster Id: 2a47436e012604919dae5fbb4fc3899c
   Zone: 1
   Management Hostname 192.168.0.139
   Storage Hostname 192.168.0.139
   #
   

6. Update the topology with the device information of the new node.

   kubectl -n kube-system exec <Heketi pod name> -- heketi-cli --user admin --secret <admin_password> device add --name=<device symlink> --node=<Node ID>
   

   Note: Add the symlink of the device as the name. For more information about how to get the symlink, see Symlinks.

   Following is a sample command and output:

   kubectl -n kube-system exec heketi-7b69bb4d48-g4kx9 -- heketi-cli --user admin --secret admin device add --name=/dev/disk/by-path/pci-0000_00_10_0 --node=5df753379b9629eda0c5eb71687ee2ea
   Device added successfully
   #