Preparing the disks

Prepare your disks for GlusterFS configuration.

You must use symlinks of the disks when you configure GlusterFS. You have the option to configure the disks as multipath volumes and to encrypt the volumes.

Multipath volumes

You can use multipath volumes for GlusterFS.

For more information about setting up multipathing, see the following documentation:

• For Ubuntu, see https://help.ubuntu.com/lts/serverguide/dm-multipath-chapter.html
• For Red Hat Enterprise Linux (RHEL), see https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/dm_multipath/index
• For SUSE Linux Enterprise Server (SLES), see https://www.suse.com/documentation/sles-12/stor_admin/data/cha_multipath.html .

Encrypted volumes

You can use encrypted volumes for GlusterFS. For more information about encrypting volumes, see Encrypting volumes by using dm-crypt.

The encrypted volumes are available at /dev/mapper/<encrypted-volume-name> location.

Symlinks

You must use the symbolic link (symlink) to identify the GlusterFS storage device. Do not use device names, such as /dev/sdb, because the name might change between system restarts.

• Use system-generated symlinks
• Use manually created symlinks

Note: The special characters that Heketi allows to be used in the device name are ^/[a-zA-Z0-9_.:/-]+$. If your device name or system-generated symlink has special characters that are not allowed by Heketi, then you must manually create the symlink.

Use system-generated symlinks

To get the symlink that the system assigns to a device, complete these steps:

1. Identify the storage devices to use. You can list the available storage devices by entering this command:

   fdisk -l
   

   The output looks like this example:

   root@icps-worker-4:~# fdisk -l
   Disk /dev/sdb: 48 GiB, 51539607552 bytes, 100663296 sectors
   Units: sectors of 1 * 512 = 512 bytes
   Sector size (logical/physical): 512 bytes / 512 bytes
   I/O size (minimum/optimal): 512 bytes / 512 bytes
   
   Disk /dev/sda: 40 GiB, 42949672960 bytes, 83886080 sectors
   Units: sectors of 1 * 512 = 512 bytes
   Sector size (logical/physical): 512 bytes / 512 bytes
   I/O size (minimum/optimal): 512 bytes / 512 bytes
   Disklabel type: dos
   Disk identifier: 0xe96e93a3
   
   Device Boot Start End Sectors Size Id Type
   /dev/sda1 * 2048 7813119 7811072 3.7G 82 Linux swap / Solaris
   /dev/sda2 7815166 83884031 76068866 36.3G 5 Extended
   /dev/sda5 7815168 83884031 76068864 36.3G 83 Linux
   

   Note: If you use multipathed or encrypted devices, these devices are available at the /dev/mapper/ location.

2. Identify devices that have at least 25 GB of storage capacity. In the example, the device /dev/sdb is considered.

3. Erase all file system, raid, and partition-table signatures by using the wipefs command. For example, to erase the signatures on device /dev/sdb, run the following command:

   sudo wipefs --all --force /dev/sdb
   

4. Get the symlink of the device by entering this command:

   ls -altr /dev/disk/*
   

   The output looks like this example:

   root@icps-worker-4:~# ls -altr /dev/disk/*
   /dev/disk/by-path:
   total 0
   drwxr-xr-x 2 root root 160 Oct 23 02:34 .
   lrwxrwxrwx 1 root root 9 Oct 23 02:34 pci-0000:00:10.0-scsi-0:0:1:0 -> ../../sdb
   lrwxrwxrwx 1 root root 9 Oct 23 02:34 pci-0000:00:10.0-scsi-0:0:0:0 -> ../../sda
   lrwxrwxrwx 1 root root 9 Oct 23 02:34 pci-0000:02:01.0-ata-1 -> ../../sr0
   drwxr-xr-x 6 root root 120 Oct 23 02:34 ..
   lrwxrwxrwx 1 root root 10 Oct 23 02:34 pci-0000:00:10.0-scsi-0:0:0:0-part2 -> ../../sda2
   lrwxrwxrwx 1 root root 10 Oct 23 02:34 pci-0000:00:10.0-scsi-0:0:0:0-part1 -> ../../sda1
   lrwxrwxrwx 1 root root 10 Oct 23 02:34 pci-0000:00:10.0-scsi-0:0:0:0-part5 -> ../../sda5
   
   /dev/disk/by-id:
   total 0
   drwxr-xr-x 2 root root 260 Oct 23 02:34 .
   lrwxrwxrwx 1 root root 9 Oct 23 02:34 wwn-0x6000c29ddc232994ce32cad1533c25e1 -> ../../sda
   lrwxrwxrwx 1 root root 9 Oct 23 02:34 wwn-0x6000c296507a2c4be3b79a74f921d283 -> ../../sdb
   lrwxrwxrwx 1 root root 9 Oct 23 02:34 scsi-36000c29ddc232994ce32cad1533c25e1 -> ../../sda
   lrwxrwxrwx 1 root root 9 Oct 23 02:34 scsi-36000c296507a2c4be3b79a74f921d283 -> ../../sdb
   
   lrwxrwxrwx 1 root root 9 Oct 23 02:34 ata-VMware_Virtual_SATA_CDRW_Drive_00000000000000000001 -> ../../sr0
   drwxr-xr-x 6 root root 120 Oct 23 02:34 ..
   lrwxrwxrwx 1 root root 10 Oct 23 02:34 wwn-0x6000c29ddc232994ce32cad1533c25e1-part2 -> ../../sda2
   lrwxrwxrwx 1 root root 10 Oct 23 02:34 wwn-0x6000c29ddc232994ce32cad1533c25e1-part1 -> ../../sda1
   lrwxrwxrwx 1 root root 10 Oct 23 02:34 scsi-36000c29ddc232994ce32cad1533c25e1-part2 -> ../../sda2
   lrwxrwxrwx 1 root root 10 Oct 23 02:34 scsi-36000c29ddc232994ce32cad1533c25e1-part1 -> ../../sda1
   lrwxrwxrwx 1 root root 10 Oct 23 02:34 wwn-0x6000c29ddc232994ce32cad1533c25e1-part5 -> ../../sda5
   lrwxrwxrwx 1 root root 10 Oct 23 02:34 scsi-36000c29ddc232994ce32cad1533c25e1-part5 -> ../../sda5
   
   /dev/disk/by-uuid:
   total 0
   lrwxrwxrwx 1 root root 9 Oct 23 02:34 2017-02-15-20-36-22-00 -> ../../sr0      
   drwxr-xr-x 6 root root 120 Oct 23 02:34 ..
   drwxr-xr-x 2 root root 100 Oct 23 02:34 .
   lrwxrwxrwx 1 root root 10 Oct 23 02:34 6d48a0d5-846b-48d3-bb0f-a626bb49c916 -> ../../sda1
   lrwxrwxrwx 1 root root 10 Oct 23 02:34 60f23142-a575-455d-8a99-6edca077d990 -> ../../sda5
   
   /dev/disk/by-label:
   total 0
   lrwxrwxrwx 1 root root 9 Oct 23 02:34 Ubuntu-Server\x2016.04.2\x20LTS\x20amd64 -> ../../sr0
   drwxr-xr-x 6 root root 120 Oct 23 02:34 ..
   drwxr-xr-x 2 root root 60 Oct 23 02:34 .
   root@icps-worker-4:~#
   

   The symlink of a device can be in any of these link paths: /dev/disk/by-path, /dev/disk/by-id, /dev/disk/by-uuid, or /dev/disk/by-label.

5. Make a note of the symlink and its link path. For the example device sdb, /dev/disk/by-id is the link path and scsi-36000c296507a2c4be3b79a74f921d283 is the symlink. For each device that you are using for GlusterFS configuration, you need to add the <link path>/<symlink> in the config.yaml file. For the example device sdb, you would add /dev/disk/by-id/scsi-36000c296507a2c4be3b79a74f921d283 in the config.yaml file.

Use manually created symlinks

In some environments, such as IBM Cloud VSI or SUSE Linux Enterprise Server (SLES), no symlinks are automatically generated for the devices. You must manually create symlinks by writing custom udev (userspace /dev) rules. When you create the symlink, use attributes that are unique to the device.

Following example includes steps to manually generate symlinks. The steps might vary with operating systems and environments.

1. Get information about the device's attributes.

   udevadm info --root --name=/dev/vdb
   

   The output resembles the following code:

   P: /devices/pci0000:00/0000:00:10.0/virtio4/block/vdb
   N: vdb
   E: DEVNAME=/dev/vdb
   E: DEVPATH=/devices/pci0000:00/0000:00:10.0/virtio4/block/vdb
   E: DEVTYPE=disk
   E: MAJOR=253
   E: MINOR=16
   E: SUBSYSTEM=block
   E: TAGS=:systemd:
   E: USEC_INITIALIZED=6705725
   E: elevator=noop
   

   Use the DEVTYPE, SUBSYSTEM, and DEVPATH attributes to create the symlink of the device.

2. Create a custom udev rules file.

   vi /lib/udev/rules.d/10-custom-icp.rules
   

   Add these lines of code to the file. Replace the attribute values with your device attribute values.

    ENV{DEVTYPE}=="disk", ENV{SUBSYSTEM}=="block", ENV{DEVPATH}=="/devices/pci0000:00/0000:00:10.0/virtio4/block/vdb" SYMLINK+="disk/gluster-disk-1"
   

3. Reload the udev rules to create the symlinks.

    udevadm control --reload-rules
    udevadm trigger --type=devices --action=change
   

4. Verify that the symlinks are created.

   ls -ltr /dev/disk/gluster-*
   

   The output resembles the following code:

   lrwxrwxrwx 1 root root 3 Jul  4 23:12 /dev/disk/gluster-disk-1 -> vdb