Configuring GlusterFS during IBM Cloud Private installation
Configure GlusterFS when you install your IBM® Cloud Private cluster.
Complete these steps to configure GlusterFS:
-
Configure a host group with nodes that you are using to configure GlusterFS. You can either use dedicated nodes for GlusterFS, or you can configure GlusterFS on nodes that are also used as worker nodes.
-
If you are using dedicated nodes to configure GlusterFS, complete these steps. These nodes are not used as worker nodes.
-
Configure a custom host group with the dedicated GlusterFS storage nodes. For more information about how to add a host group, see Defining custom host groups.
Following is an example configuration of a host group with dedicated GlusterFS storage nodes. You add this configuration in the
/<installation_directory>/cluster/hosts
file.Note: The worker nodes and the GlusterFS nodes are not the same.
[worker] 2.2.2.2 ... 2.2.2.9 . . [hostgroup-glusterfs] 6.6.6.6 ... 6.6.6.9
-
-
If you are configuring GlusterFS on nodes that are also used as worker nodes, complete these steps.
- In the
/<installation_directory>/cluster/hosts
file, add the same nodes as worker nodes and as a custom host group. -
In the
/<installation_directory>/cluster/config.yaml
file, configureno_taint_group
parameter with the custom host group name.
Following is an example configuration of shared storage nodes. You add this configuration in the/<installation_directory>/cluster/hosts
file.[worker] 2.2.2.2 ... 2.2.2.9 . . [hostgroup-glusterfs] 2.2.2.2 ... 2.2.2.9
Following is an example configuration of adding the
no_taint_group
parameter. You add this configuration in the/<installation_directory>/cluster/config.yaml
file.no_taint_group: ["hostgroup-glusterfs"]
- In the
-
-
If firewall is enabled, add the list of required ports to the
/<installation_directory>/cluster/config.yaml
file. Locate the sectionfirewall_enabled: true
. Add the following ports for the custom host group that you created with dedicated GlusterFS storage nodes.
Following is an example configuration of the custom host grouphostgroup-glusterfs
:firewall_open_ports: hostgroup-glusterfs: - 24007/tcp - 24008/tcp - 2222/tcp - 49152-49251/tcp
Note: If you use existing worker nodes to deploy GlusterFS, you must manually open all the ports on all the nodes.
-
Enable GlusterFS Storage. Set
storage-glusterfs: enabled
under the management services list in the/<installation_directory>/cluster/config.yaml
file.management_services: istio: disabled vulnerability-advisor: disabled storage-glusterfs: enabled storage-minio: disabled
-
Add the following piece of code to the
config.yaml
file:
## GlusterFS Storage Settings
storage-glusterfs:
nodes:
- ip: <worker_node_m_IP_address>
devices:
- <link path>/<symlink of device aaa>
- <link path>/<symlink of device bbb>
- ip: <worker_node_n_IP_address>
devices:
- <link path>/<symlink of device ccc>
- ip: <worker_node_o_IP_address>
devices:
- <link path>/<symlink of device ddd>
storageClass:
create: true
name: glusterfs
isDefault: false
volumeType: replicate:3
reclaimPolicy: Delete
volumeBindingMode: Immediate
volumeNamePrefix: icp
additionalProvisionerParams: {}
allowVolumeExpansion: true
gluster:
resources:
requests:
cpu: 500m
memory: 512Mi
limits:
cpu: 1000m
memory: 1Gi
heketi:
backupDbSecret: heketi-db-backup
authSecret: "heketi-secret"
maxInFlightOperations: "20"
dbSyncupDelay: "10"
tls:
generate: true
issuer: "icp-ca-issuer"
issuerKind: "ClusterIssuer"
secretName: ""
resources:
requests:
cpu: 500m
memory: 512Mi
limits:
cpu: 1000m
memory: 1Gi
nodeSelector:
key: hostgroup
value: glusterfs
prometheus:
enabled: true
path: "/metrics"
port: 8080
tolerations: []
podPriorityClass: "system-cluster-critical"
Following are the parameter descriptions. For a list of available parameters, see Configuration .
Note: If you are configuring the default storage class and volume type parameters, ensure that you use the correct syntax. Use isDefault: <true or false>
and volumeType: <volume type>
.
ip
is the IP address of the node on which you are configuring GlusterFS.-
devices
is the full path to the symlink of the storage device.Note: Do not add the device name. Add the Symlinks.
-
storageClass
creates a storage class for GlusterFS.create
is the option to configure a storage class. The default value istrue
. If you do not want to create a storage class, specifyfalse
.name
is the storage class name. The default value isglusterfs
. The storage class name must comply with the Kubernetes naming convention: up to 253 characters and only lowercase alphanumeric characters,-
, and.
.isDefault
is the option to make the GlusterFS storage class as the default storage class. The default value isfalse
. If you want this storage class to be the default, specifytrue
.volumeType
is a configuration parameter for the volume. You can set this value toreplicate:3
, which creates a replica volume of count 3. The valid parameter values are:none
;replicate:<replicate_count>
;disperse:<data>:<redundancy_count>
. The default value isreplicate:3
.
Note: If you are configuring GlusterFS on a single node, setvolumeType: none
.reclaimPolicy
is the GlusterFS storage class reclaim policy. Default value isDelete
.volumeBindingMode
is the GlusterFS storage class volume binding mode. Default value isImmediate
.volumeNamePrefix
is the GlusterFS storage class volume name prefix. Default value isicp
.additionalProvisionerParams
are extra storage class provisioner parameters.-
allowVolumeExpansion
is to define whether expansion of volume is allowed. Default value istrue
.Note: At any point in time later, you can create more storage classes to use GlusterFS storage. See Creating a storage class for GlusterFS.
-
gluster
sets the parameters for GlusterFS configuration.resources
defines the maximum and minimum CPU and memory that is required.requests
cpu
is the minimum amount of CPU required. Default value is 500 millicpu (m).memory
is the minimum amount of memory required. Default value is 512 Mi.
limits
cpu
is the maximum amount of CPU allowed. Default value is 1000m.memory
is the maximum amount of memory allowed. Default value is 1Gi.
heketi
sets the parameters for Heketi configuration.backupDbSecret
is the Heketi database to be backed up to a kubernetes secret. Default value isheketi-db-backup
.authSecret
is the secret that has the encrypted password of the Heketi user 'admin'. Note: The secret is automatically created by using the<default_admin_password>
that is added in the/<installation_directory>/cluster/config.yaml
file.maxInFlightOperations
is the maximum number of simultaneous requests for persistent volume. The default value is 20.dbSyncupDelay
is the delay in seconds to sync up data in the Heketi database with backup secret. The default value is 10.tls
sets transport layer security (TLS) configuration parameters.generate
is a Boolean flag. When set totrue
, the flag creates a certificate by using the IBM Cloud Private certificate authority (CA). If you do not set the flag, you must create a secret that contains a private key, TLS certificate, and a CA certificate. You must provide the secret name in theheketi.tls.secretName
parameter.issuer
is the name of the IBM Cloud Private certificate issuer.issuerKind
is the kind of CA from which the signed x509 certificates is obtained. Valid values areClusterIssuer
andIssuer
.secretName
is the certificate that you provide as secret object. The name of key must betls.key
and the name of the certificate must betls.crt
. You must set this parameter when you set theheketi.tls.generate
flag tofalse
.
resources
is the maximum and minimum CPU and memory that is required.requests
cpu
is the minimum amount of CPU required. Default value is 500m.memory
is the minimum amount of memory required. Default value is 512Mi.
limits
cpu
is the maximum amount of CPU allowed. Default value is 1000m.memory
is the maximum amount of memory allowed. Default value is 1Gi.
prometheus
sets the option to enable or disable prometheus monitoring.
Note: Prometheus provides a dashboard of statistics about health and usage of GlusterFS. If you want to use Prometheus for GlusterFS, you must enable it now. You cannot enable it after your IBM Cloud Private cluster is installed.enabled
is the setting to enable or disable prometheus monitoring. Default value istrue
.path
is the Heketi path to pull the metrics. Default value is/metrics
.port
is the port on which the Heketi service is exposed. Default value is8080
.
nodeSelector
sets the node selector label for the storage resources. Use the name of the custom host group that you created for GlusterFS configuration. For example, the custom host grouphostgroup-glusterfs
creates nodes with the labelhostgroup=glusterfs
.key
is the label key. Default value ishostgroup
.value
is the label value. Default value isglusterfs
.
podPriorityClass
is the priority class preference for the GlusterFS and Heketi pods.-
tolerations
is the list of tolerations that you want to specify whether the storage nodes have additional taints. -
Configuration of GlusterFS is complete. Proceed with the IBM Cloud Private installation.