Deployment considerations

Ensure that the following steps are completed before you deploy IBM Spectrum Scale Container Storage Interface driver in your cluster.

Worker nodes selection: By default, Kubernetes or Red Hat OpenShift® schedules the IBM Spectrum Scale Container Storage Interface driver pods on all worker nodes. It is essential to have IBM Spectrum Scale client installed on all these nodes. If you want to schedule the IBM Spectrum Scale Container Storage Interface driver pods only on selected worker nodes, you must label the selected nodes and use this label in node selector. For more information, see Using the node selector.
Node selection for StatefulSets: CSI external attacher and CSI external provisioner are sidecar containers that run as two separate StatefulSets. These pods can be scheduled on any of the worker nodes by Kubernetes. As a best practice, it is recommended to run these pods on two separate stable nodes. The StatefulSets by design of Kubernetes do not automatically fail over to another node, hence it is recommended to schedule them to run on reliable nodes. On Red Hat® OpenShift, if the infrastructure nodes are worker nodes, it is recommended to schedule the sidecar containers to run on infrastructure nodes. Scheduling them to run on specific nodes can be achieved by using nodes labels and nodeSelectors. For more information, see Using the node selector. IBM Spectrum Scale Container Storage Interface driver pod must also be scheduled on the nodes that run StatefulSets.
Remote cluster setup: If you plan to use remotely mounted file system for PVC provisioning, ensure the following points:
- IBM Spectrum Scale GUI is initialized and running on both clusters (owning cluster and accessing cluster)
- Remote cluster details are added to the Operator configuration. For more information, see Remote cluster support.
SELinux considerations: Different Kubernetes distributions handle the SELinux enforcing mode differently. There might be differences in terms of SELinux context that is set on files, relabeling of volumes and the process context of containers. As a prerequisite, appropriate SELinux rules must be set up to allow IBM Spectrum Scale Container Storage Interface driver containers to access the required resources on host. For example, “container_t” context needs to have access to csi.sock and the IBM Spectrum Scale file system, or the files that need access from containers need to have the “container_file_t” context set. Refer to audit logs for any SELinux failures and set up appropriate rules as required.
Node names: At times, it is possible that IBM Spectrum Scale cluster and Kubernetes/Red Hat OpenShift cluster are configured with different node names for the same host. Use the mmlscluster and kubectl get nodes commands and check the node names of IBM Spectrum Scale cluster and Kubernetes cluster. If the names are different, then configure node mapping in the Operator configuration. For more information, see Kubernetes to IBM Spectrum Scale node mapping.
Note: Node names must match with the output of the mmlsmount <fsname> -L command, or the curl command curl --insecure -u 'username:password' -X GET https://guiHost:443/scalemgmt/v2/filesystems/<filesystemname>?fields=mount.
Internet connectivity: If your worker nodes have internet connectivity and access to the quay.io registry, the IBM Spectrum Scale Container Storage Interface driver downloads and uploads the required images automatically during deployment. Otherwise, you need to manually download the following images:
- quay.io/ibm-spectrum-scale/ibm-spectrum-scale-csi-driver:v2.1.0
- quay.io/ibm-spectrum-scale/ibm-spectrum-scale-csi-operator:v2.1.0
- us.gcr.io/k8s-artifacts-prod/sig-storage/csi-node-driver-registrar:v2.0.1
- us.gcr.io/k8s-artifacts-prod/sig-storage/csi-attacher:v3.0.0
- us.gcr.io/k8s-artifacts-prod/sig-storage/csi-provisioner:v2.0.2
Note: Upload these images on all worker nodes by using docker/podman command. In case you use a private registry for your Red Hat OpenShift/Kubernetes environment, upload the images to your private registry. Use the new image URL and registry credentials in the imagePullSecrets section of the csiscaleoperators.csi.ibm.com_cr.yaml file. For more information, see the following topics:
- To create secrets by using private registry credentials, see Pull an Image from a Private Registry
- For information on the imagePullSecrets section, see the csiscaleoperators.csi.ibm.com_cr.yaml file on the Operator page.
IBM Spectrum® Scale services: As a best practice, it is recommended that GUI nodes, protocol nodes, and NSD nodes are not part of the Kubernetes cluster.