Limitations of protocols on remotely mounted file systems

You must consider certain restrictions when you plan on setting up a multi-cluster protocol environment.

Refer to the following points:
  • You can configure one storage cluster and up to five protocol clusters (current limit).
  • The storage cluster owns all of the exported IBM Spectrum Scale™ file systems. This means at least two file systems per protocol cluster (one CES shared root + one data file system)
  • The storage cluster must not have any protocol nodes (CES must be disabled).
  • The protocol clusters cannot own any IBM Spectrum Scale file systems, only remote mounts from the storage cluster are allowed.
  • Any file system can be remotely mounted by exactly one protocol cluster. Sharing a file system between multiple protocol clusters might cause data inconsistencies.
  • The primary use case for multi-cluster protocol is to allow multiple authentication configurations. The setup must not be used for extending the scalability of Cluster Export Services (CES) or to work around defined limitations (for example, number of SMB connections).
  • This setup provides some level of isolation between the clusters, but there is no strict isolation of administrative operations, and there is no guarantee that administrators on one cluster cannot see data from another cluster. Strict isolation is guaranteed through NFS or SMB access only.
  • Each protocol cluster must use a dedicated file system, it is not allowed to share a file system between multiple protocol clusters.
  • Storage and protocol clusters are in the same site/location, high network latencies between them can cause problems.
  • This setup cannot be used for Object or iSCSI services.
  • Due to the separation of duties (resource clusters own the file systems and protocol clusters own the NFS/SMB exports), certain management task must be done in the corresponding cluster:
    • File system related operations (like creating file systems, filesets, creating snapshots) must be done in the resource cluster.
    • Export-related operations (creating exports, managing CES IPs, managing authentication and ACLs) must be done in the protocol cluster.
    Note: This also means that certain operations such as creation of fileset and snapshots do not work on the GUI.
  • Cross-protocol change notifications will not work on remotely-mounted file systems. For example, if an NFS client changes a file, the system will not issue a "file change" notification to the SMB client which has asked for a notification.
Parent topic: Using NFS/SMB protocol over remote cluster mounts