Enabling CES and protocols

Configure the setup for CES and protocols on AWS and Microsoft Azure clusters by using IBM Storage Scale cloudkit.

For a detailed explanation about protocols, see Protocols support overview: Integration of protocol access methods with GPFS.

Cloudkit provides a simplified way to provision protocol or CES instances with virtual or alias IP addresses. To provision these instances, cloudkit uses AWS and Microsoft Azure IP addresses that are secondary and private, combined with a callback within IBM Storage Scale clusters. This method contributes to efficiently handle the CES address failover if protocol or CES node failures occur.

To enable the protocol on the cluster, issue the next command.
./cloudkit edit cluster --role "Protocol-only"

The edit cluster operation provisions the requested number of protocol nodes along with the secondary private or alias IP addresses that can be further used as CES IP addresses. The number of secondary private IP addresses is autocalculated from the free pool of the subnet and are equal to the number of protocol nodes.

After the protocol nodes are added to the cluster, run the following command for protocol services configuration.
./cloudkit setup protocol

This command configures CES shared root and a list of protocol services to be set up such as NFS, SMB, and S3.

In AWS, to allow protocol client nodes to reach the CES IPs, update the associated route table with the protocol client subnet to add routes for CES IPs. These CES IPs must point to corresponding network interfaces.

To update the route table, complete the following steps:

  1. Retrieve the route table ID by specifying an AWS region and a client subnet ID.
    ROUTE_TABLE_ID=$(aws ec2 describe-route-tables --region <aws-region> --filters "Name=association.subnet-id,Values=<subnet-id>" --query "RouteTables[0].RouteTableId" --output text)
  2. Add a route for the CES IPs by specifying an AWS region, CES IPs, and the network interface for the protocol node.
    aws ec2 create-route --region <aws-region> --route-table-id $ROUTE_TABLE_ID --destination-cidr-block <CES_IP>/32 --network-interface-id <protocol-node-eni-id>

For multiple protocol nodes, repeat step 2 to create routes for all protocol nodes with their respective CES IPs and network interface IDs (eni-IDs).

Remember: During the service initialization, you must manually authenticate and configure the fileset according to the requirement.
Note: In the Throughput-Performance-Persistent-Storage mode, cloudkit creates a file system by provisioning gp3 with throughput, thus saturating the instance bandwidth. Based on your application requirement, modify the IOPS per disk.

In higher network bandwidth instances, you need to adjust maxTcpConnsPerNodeConn to a higher value (between 2-8).