Remote cluster support with IBM Storage Scale native REST API

IBM Storage Scale native REST API provides remote cluster support. With this support, you can mount IBM Storage Scale file systems from one IBM Storage Scale cluster to a remote IBM Storage Scale cluster.

In classic IBM Storage Scale clusters, the ability to access and mount file systems that are owned by other IBM Storage Scale clusters is done by using the mmauth, mmremotecluster, and mmremotefs commands. In the native REST API environment, the scalectl cluster remote and scalectl filesystem remote or corresponding REST APIs are introduced to access and mount file systems that are owned by other IBM Storage Scale clusters. For more information, see scalectl cluster command, scalectl filesystem command, or Remote cluster endpoints.

Know the definition of the following terms that are used in the remote cluster environment:

Remote cluster: IBM Storage Scale cluster that is defined as a remote cluster in the local cluster. A remote cluster is either an accessing cluster, an owning cluster, or both.
Accessing cluster: IBM Storage Scale cluster that is accessing a file system that is owned by another cluster.
Owning cluster: IBM Storage Scale cluster that owns a file system and is exporting the file system to one or more clusters.

For example, if Cluster A owns the fs1 file system and Cluster B mounts fs1 file system, then:

Cluster A is the owning cluster. Cluster A authorizes Cluster B to mount thefs1 file system.
Cluster B is the accessing cluster. Cluster B requests access to the fs1 file system from Cluster A.

Figure 1. High-level architecture of remote cluster setup

Preconfiguration

Key exchange protocol: To establish a secure remote cluster relationship, a secure key exchange protocol must be in place. During the cluster remote add operation, the accessing cluster sends accessing cluster keys to the owning cluster. The owning cluster receives accessing cluster keys, verifies accessing cluster identity, and verifies that the accessing cluster is authorized. If authorized, then the owning cluster stores accessing cluster keys for communication and replies with owning cluster keys. Then, the accessing cluster stores owning cluster keys for further communication.
Mutual TLS (mTLS): Communication between clusters uses mTLS for secure authentication and data integrity. Both clusters must trust each other’s certificate authority (CA). The owning cluster validates the accessing cluster by verifying that the cluster name is present in the accessing cluster's certificate CN (Common Name) field during the cluster remote authorize operation. Similarly, the accessing cluster validates the owning cluster during the cluster remote add operation.
Node identities: Each node must have an X.509 certificate for communication across scaleadmd services. Node identities are managed on a per node basis through the scalectl node config command. For more information, see Node identities.

Configuration

To mount a remote file system from one IBM Storage Scale cluster to another IBM Storage Scale cluster, complete the following steps:

If required, create an accessing cluster.

scalectl cluster create -N <NodeName>  --cluster-name "<ClusterName>" -A

Authorize the accessing cluster on the owning cluster. Issue the following command on the owning cluster:

scalectl cluster remote authorize --name <AccessingClusterName> --cluster-id <3793952426292428382> --file resourcefile.json

A sample resourcefile.json contains the following information:

{
  "version": 0,
  "resources": {
    "filesystems": [
      {
        "name": "fs1",
        "disposition":"GRANT",
        "access_type": "rw"
      }
      ],
    "filesets": [
        {
          "name": "root",
          "fs_name": "fs1",
          "disposition": "GRANT"
        },
        {
          "name": "fileset1",
          "fs_name": "fs1",
          "disposition": "GRANT"
        },
        {
          "name": "fileset2",
          "fs_name": "fs1",
          "disposition": "GRANT"
        }
      ]
  }
}

Add the owning cluster definition on the accessing cluster. Issue the following command on the accessing cluster:
```
scalectl cluster remote add --name <OwningingClusterName> --contact-nodes node1
```

Add the remote file system definition on the accessing cluster. Issue the following command on the accessing cluster:

scalectl filesystem remote add --name <DeviceName> -T <MountPoint> --remote-name <FileSystemName> --owning-cluster <OwningClusterName>

Mount the file system. Issue the following command on the accessing cluster:
```
scalectl filesystem mount <FilesystemName> -T <MountPoint>
```

Table 1 lists the commands and their corresponding native REST APIs, along with the types of clusters they can be executed on:

Table 1. Supported cluster type for commands and REST APIs
Command	REST API	Cluster type
scalectl cluster remote authorize	/scalemgmt/v3/clusters/remote/authorized: POST	Owning cluster only
scalectl cluster remote unauthorize	/scalemgmt/v3/clusters/remote/authorized/{name}: DELETE	Owning cluster only
scalectl cluster remote add	/scalemgmt/v3/clusters/remote: POST	Accessing cluster only
scalectl cluster remote delete	/scalemgmt/v3/clusters/remote/{name}: DELETE	Accessing cluster only
scalectl cluster remote refresh	/scalemgmt/v3/clusters/remote/{name}/refresh: POST	Accessing cluster only
scalectl cluster remote update	/scalemgmt/v3/clusters/remote/{name}: PATCH	Both owning and accessing clusters
scalectl cluster remote list	/scalemgmt/v3/clusters/remote: GET	Both owning and accessing clusters
scalectl cluster remote get	/scalemgmt/v3/clusters/remote/{name}: GET	Both owning and accessing clusters