Remote cluster support

IBM Spectrum Scale provides a feature to mount IBM Spectrum Scale file systems from one IBM Spectrum Scale cluster (owning cluster) to another IBM Spectrum Scale cluster (accessing cluster). You can configure an IBM Spectrum Scale Container Storage Interface driver to work with a remotely mounted IBM Spectrum Scale.

The cluster that owns the file system is responsible for administering the file system and granting access to other clusters on a per-cluster basis. After access to a file system is granted to nodes in another IBM Spectrum Scale cluster, the nodes can mount the file system and do data operations as if the file systems are locally owned.

For more information on the remote mount setup, see Accessing a remote GPFS file system in the IBM Spectrum Scale: Command and Programming Reference.

Primary cluster is IBM Spectrum Scale cluster where Red Hat® OpenShift® or Kubernetes worker nodes coexist with IBM Spectrum Scale client nodes. In this example deployment, cluster A is designated as the primary cluster.

The cluster O is another IBM Spectrum Scale cluster that has two file systems gpfs and fs1. The file system gpfs is mounted on Cluster A as file system mygpfs while file system fs1 is not exposed to Cluster A.

For each IBM Spectrum Scale cluster, cluster entry must be added under the clusters section of the custom resource.

- id: "<cluster id of IBM Spectrum Scale>”
  restApi:
  - guiHost: "< FQDN or IP GUI node of IBM Spectrum Scale cluster>"
  secrets: "<secret name for GUI of IBM Spectrum Scale cluster>"
  secureSslMode: false

One IBM Spectrum Scale cluster must be the primary cluster for IBM Spectrum Scale Container Storage Interface driver deployment. Primary cluster is marked by adding the primary section in the respective cluster entry. In the example described in the figure, deployment Cluster A is the primary cluster. Entry for primary cluster looks as follows:

- id: "<cluster id of IBM Spectrum Scale Cluster which is Primary cluster >”
     primary: 
	 primaryFs: <name of primary filesystem>
     restApi:
     - guiHost: "< FQDN or IP of Primary Spectrum Scale cluster’s GUI node>"
     secrets: "<secret name for GUI of Primary Spectrum Scale cluster >”
     secureSslMode: false

In the example deployment, there are two IBM Spectrum Scale clusters, hence two entries of clusters are added, one for the primary cluster (Cluster A) and another one for cluster O (Owning cluster).

The custom resource configuration slightly changes based on whether the primary file system is locally owned (gpfs0 in the example deployment) or remotely mounted ( mygpfs in the example deployment). The changes are in the primary section of the primary cluster entry.

The custom resource for the example deployment when primaryFS is a locally owned file system (gpfs0) looks as follows:

---
apiVersion: csi.ibm.com/v1
kind: "CSIScaleOperator"
metadata:
  name: "ibm-spectrum-scale-csi"
  namespace: "ibm-spectrum-scale-csi-driver"
  labels:
    app.kubernetes.io/name: ibm-spectrum-scale-csi-operator
    app.kubernetes.io/instance: ibm-spectrum-scale-csi-operator
    app.kubernetes.io/managed-by: ibm-spectrum-scale-csi-operator
    release: ibm-spectrum-scale-csi-operator
status: {}
spec:
  scaleHostpath: "/ibm/gpfs0"
  clusters:
    - id: "<cluster id of IBM Spectrum Scale Cluster A>"
      secrets: "guisecretNode1"
      secureSslMode: false
      primary:
        primaryFs: "gpfs0"
      restApi:
        - guiHost: "<FQDN/IP of Node 1>"
    - id: "<cluster id of IBM Spectrum Scale Cluster O>"
      secrets: "guisecretNodeA"
      secureSslMode: false
      restApi:
        - guiHost: "<FQDN/IP of Node A>"
  attacherNodeSelector:
    - key: "scale"
      value: "true"
  provisionerNodeSelector:
    - key: "scale"
      value: "true"
  pluginNodeSelector:
    - key: "scale"
      value: "true"
---

---
apiVersion: csi.ibm.com/v1
kind: "CSIScaleOperator"
metadata:
  name: "ibm-spectrum-scale-csi"
  namespace: "ibm-spectrum-scale-csi-driver"
  labels:
    app.kubernetes.io/name: ibm-spectrum-scale-csi-operator
    app.kubernetes.io/instance: ibm-spectrum-scale-csi-operator
    app.kubernetes.io/managed-by: ibm-spectrum-scale-csi-operator
    release: ibm-spectrum-scale-csi-operator
status: {}
spec:
  scaleHostpath: "/ibm/mygpfs"
  clusters:
    - id: "<cluster id of IBM Spectrum Scale Cluster A>"
      secrets: "guisecretNode1"
      secureSslMode: false
      primary:
        primaryFs: "mygpfs"
	remoteCluster: "<cluster id of IBM Spectrum Scale Cluster O (Owning cluster)>"
      restApi:
        - guiHost: "<FQDN/IP of Node 1>"
    - id: "<cluster id of IBM Spectrum Scale Cluster O(owning cluster)>"
      secrets: "guisecretNodeA"
      secureSslMode: false
      restApi:
        - guiHost: "<FQDN/IP of Node A>"
  attacherNodeSelector:
    - key: "scale"
      value: "true"
  provisionerNodeSelector:
    - key: "scale"
      value: "true"
  pluginNodeSelector:
    - key: "scale"
      value: "true"
---