Deploying IBM Spectrum Scale on AWS

Follow these given steps to deploy IBM Spectrum Scale™ on AWS:

Prepare your AWS account:
1. Create an AWS account at AWS, if you do not have one.
2. Use the region selector in the navigation bar to choose the AWS region where you want to deploy IBM Spectrum Scale on AWS.
3. Create a key pair in your preferred region. For information on how to create a key pair, see Amazon EC2 Key Pairs
4. If necessary, request a service limit increase for the EC2 instance types that you intend to deploy. To do this, in the AWS Support Center, choose Create Case > Service Limit Increase > EC2 instances, and then complete the fields in the Limit Increase form.
Launch the IBM Spectrum Scale AWS stack:
1. Choose one of the following options to launch the AWS CloudFormation template into your AWS account.
  - Option 1: Deploying IBM Spectrum Scale on a new Amazon VPC with a single availability zone
  - Option 2: Deploying IBM Spectrum Scale on a new Amazon VPC with multiple availability zones
  - Option 3: Deploying IBM Spectrum Scale on an existing Amazon VPC.
    For a single availability zone deployment configuration, IBM Spectrum Scale does not replicate any of the file system data. It is recommended that this option only be used for cases in which a higher probability of data loss is acceptable in order to potentially improve performance and save on data movement costs. For example, in a scratch file system scenario.
    
    To better tolerate EBS failures, it is recommended to use the multiple availability zone option, in which IBM Spectrum Scale replicates the data such that there are two total copies of each data element. In this case, the replication is intended to deal with a loss of a volume. However, the probability of data loss depends on how the failure rates impact the total number of volumes that might fail at any given point. It is recommended that users carefully read Amazon's statement regarding the durability of EBS volumes, particularly the annual failure rates, in order to better assess the possibility of potential data loss resulting from EBS volume failures. For more information, see Amazon EBS features.
    
    Note: This is where the network infrastructure for IBM Spectrum Scale is built. The template is launched in the US East (Ohio) region by default. You can change the region.
2. On the Select Template page, keep the default setting for the Template URL, and then choose Next.
3. On the Specify Details page, change the stack name if needed. Review the parameters for the template.
  Note: Provide values for the parameters that require input. For all other parameters, review the default settings and customize them as necessary.
4. In the Options page, you can specify tags or key-value pairs for resources in your stack, and set Advanced options.
  Note: For more information on how to specify tags, see AWS CloudFormation Resource Tags Type. For setting stack options in the Advanced options section, see Setting AWS CloudFormation Stack Options.
5. In the Review page, review and confirm the template settings. Under Capabilities, select the check box to acknowledge that the template creates IAM resources.
6. Click Create to deploy the stack.
7. Monitor the status of the stack. When the status displays CREATE_COMPLETE, the IBM Spectrum Scale environment is ready.
8. View the resources that were created for the stack in the URLs displayed in the Outputs tab.
Note: This IBM Spectrum Scale AWS stack deployment is automated by the nested AWS CloudFormation templates. The main template builds the network-related resources first, using the VPC template, and then launches separate stacks for the bastion host and IBM Spectrum Scale cluster. Deleting the stack created by the main template deletes the entire IBM Spectrum Scale deployment stack. However, you still need to delete CloudWatch alerts manually.
Connect to the IBM Spectrum Scale cluster.
When the AWS CloudFormation template has successfully created the stack, all instances of compute and NSD servers launched by the AWS set up is up and running with the IBM Spectrum Scale file system mounted on it.
To connect to the IBM Spectrum Scale cluster take the following steps:
1. Open the Amazon EC2 console at https://console.aws.amazon.com/ec2/.
2. In the navigation pane, under Instances, check for the public DNS (IPv4) value for the instance named LinuxBastion.
3. Use your AWS private key to connect to the bastion host using SSH.
  Note: This is the key that you specify in the Key Name parameter of the AWS CloudFormation template during deployment.
4. From the bastion host, use the SSH agent to log in to any of the compute instances or NSD server instances that were launched by the AWS CloudFormation templates.
  Note: For more information about using an SSH agent to forward your private key on connection, see Using SSH agent forwarding.
  
  Important: Do not copy your private key to the bastion host instance.
  
  Figure 1. High-level IBM Spectrum Scale cluster architecture to connect from host

Test the deployment using IBM Spectrum Scale commands

After you log in to a compute or NSD server instance, you can administer IBM Spectrum Scale. For more information on administering IBM Spectrum Scale, see Administering.

Note: You must be a root user to run IBM Spectrum Scale administration commands.

The mmlscluster command displays the details of the IBM Spectrum Scale cluster. The command gives an output similar to the following:

[ec2-user@ip-10-0-1-110]$ /usr/lpp/mmfs/bin/mmlscluster 

GPFS cluster information
========================
  GPFS cluster name:         ip-10-0-1-110.ap-south-1.compute.internal
  GPFS cluster id:           12901386493707864068
  GPFS UID domain:           ip-10-0-1-110.ap-south-1.compute.internal
  Remote shell command:      /usr/bin/ssh
  Remote file copy command:  /usr/bin/scp
  Repository type:           CCR

 Node  Daemon node name                           IP address  Admin node name                            Designation
---------------------------------------------------------------------------------------------------------------------
   1   ip-10-0-1-110.ap-south-1.compute.internal  10.0.1.110  ip-10-0-1-110.ap-south-1.compute.internal  quorum-manager-perfmon
   2   ip-10-0-3-42.ap-south-1.compute.internal   10.0.3.42   ip-10-0-3-42.ap-south-1.compute.internal   quorum-manager-perfmon
   3   ip-10-0-1-72.ap-south-1.compute.internal   10.0.1.72   ip-10-0-1-72.ap-south-1.compute.internal   quorum-perfmon
   4   ip-10-0-3-82.ap-south-1.compute.internal   10.0.3.82   ip-10-0-3-82.ap-south-1.compute.internal   perfmon

For more information on the mmlscluster command, see the IBM Spectrum Scale: Command and Programming Reference guide in the IBM Spectrum Scale Knowledge Centre.

The mmlsnsd command displays the NSD server information. The command gives an output similar to the following:

[ec2-user@ip-10-0-1-110]$ /usr/lpp/mmfs/bin/mmlsnsd 

 File system   Disk name    NSD servers                                    
---------------------------------------------------------------------------
 fs1           0a9503b2cf264bf08 ip-10-0-3-42.ap-south-1.compute.internal 
 fs1           0d7e3d725140a0cdb ip-10-0-1-110.ap-south-1.compute.internal 
 fs1           0fb10f7d9a981f39e ip-10-0-1-72.ap-south-1.compute.internal

For more information on the mmlsnsd command, see the IBM Spectrum Scale: Command and Programming Reference guide in the IBM Spectrum Scale Knowledge Centre.

The mmlsdisk command displays disk details. The command gives an output similar to the following:

[ec2-user@ip-10-0-1-110]$ mmlsdisk fs1 -L
disk         driver   sector     failure holds    holds                                    storage
name         type       size       group metadata data  status        availability disk id pool         remarks   
------------ -------- ------ ----------- -------- ----- ------------- ------------ ------- ------------ ---------
0a9503b2cf264bf08 nsd         512           1 Yes      Yes   ready         up                 1 system        desc
0d7e3d725140a0cdb nsd         512           2 Yes      Yes   ready         up                 2 system        desc
0fb10f7d9a981f39e nsd         512           3 No       No    ready         up                 3 system        desc
Number of quorum disks: 3 
Read quorum value:      2
Write quorum value:     2

For more information on the mmlsdisk command, see the IBM Spectrum Scale: Command and Programming Reference guide in the IBM Spectrum Scale Knowledge Centre.

The mmdf command displays the fs1 filesystem information. The command gives an output similar to the following:

[ec2-user@ip-10-0-1-110]$ mmdf fs1
disk                disk size  failure holds    holds           free in KB          free in KB
name                    in KB    group metadata data        in full blocks        in fragments
--------------- ------------- -------- -------- ----- -------------------- -------------------
Disks in storage pool: system (Maximum disk size allowed is 3.97 TB)
0a9503b2cf264bf08     524288000    1   Yes      Yes       520880128 ( 99%)          8792 ( 0%) 
0d7e3d725140a0cdb     524288000    2   Yes      Yes       520880128 ( 99%)          8792 ( 0%) 
0fb10f7d9a981f39e       5242880    3   No       No                0 (  0%)             0 ( 0%) 
                -------------                         -------------------- -------------------
(pool total)       1053818880                            1041760256 ( 99%)         17584 ( 0%)

                =============                         ==================== ===================
(total)            1053818880                            1041760256 ( 99%)         17584 ( 0%)

Inode Information
-----------------
Number of used inodes:            4038
Number of free inodes:          497722
Number of allocated inodes:     501760
Maximum number of inodes:      1025024

For more information on the mmdf command, see the IBM Spectrum Scale: Command and Programming Reference guide in the IBM Spectrum Scale Knowledge Centre.

The mmgetstate command shows the state of the instances in the cluster. The command gives an output similar to the following:

[ec2-user@ip-10-0-1-110]$ sudo /usr/lpp/mmfs/bin/mmgetstate -a

 Node number  Node name        GPFS state  
-------------------------------------------
       1      ip-10-0-1-110    active
       2      ip-10-0-3-42     active
       3      ip-10-0-1-72     active
       4      ip-10-0-3-82     active

For more information on the mmgetstate command, see the IBM Spectrum Scale: Command and Programming Reference guide in the IBM Spectrum Scale Knowledge Centre.