Binding your file system or fileset to the Cloud service by creating a container pair set

Up to this point, the configuration work was about creating access out to the cloud. Now, you need to bind all this to the data on your cluster, and you do that by creating a container pair set that associates your file system or fileset to the Cloud service. Once you create your container pair set, your Cloud service is usable.

Cloud services internally creates two containers on cloud storage for storing data as well as meta-data. However, some cloud providers require containers to be created using its native interfaces. In that case, you need to provide the names. The containers that are created for Cloud data sharing can be shared with other file systems or Cloud services. However, the containers that are created for tiering cannot be shared. Creating the container pair set is how you bind a file system to a Cloud service. Note that all file sets being bound to a file system must be assigned to the same Cloud services node class.
Note: Start of changeWhen the existing tiering container reaches default 100,000,000 (100 million) entries or configured non-default threshold, and if auto-spillover is enabled, the Cloud service automatically creates a new container during the next reconcile operation. As an administrator, you can still create a new container for the same path, even before reaching the spillover threshold. After a new container is created for the configured filesystem or fileset path or auto-spillover, the previous container goes to the Inactive state and new migrations go to the newly created container. Creation of a new container only affects target container for new migrations. Recalls are unaffected and continue from the container (including inactive containers) where data was migrated. If auto-spillover (new container creation) fails for any reason, the current active container indicates overdue status for container spillover.End of change

You must decide whether or not to add encryption to the container at the time of creating it, and accordingly use the KeyManagerName parameter. If you create a container pair without a key manager, you will not be able to add encryption to the container at a later point by using the mmcloudgateway ContainerPairSet update command.

Note: In this release, containers do not have encryption enabled. Hence, administrators need to explicitly add "--enc ENABLE" parameter while creating a container pair set, to ensure that the data is encrypted while being tiered to a cloud storage.
If you have applications that frequently access the front end of the file, you might want to consider enabling thumbnail support. An example of an application that accesses the first few bytes of a file is Windows Explorer in order to provide a thumbnail view of image files. There is no limit on the amount of data you can cache as the appropriate cache size is application specific. You can create a container pair set with thumbnail enabled, and the scope can be enabled to either a file system or a fileset according to your business requirements.
Note:
  • Changing the mount point for a file system or the junction path of a fileset after it is associated with a container is not supported.
  • You can enable or disable transparent recall policy by using the --transparent-recalls {ENABLE|DISABLE} parameter. However, this parameter is optional, and transparent recall policy is enabled by default even if you do not use this parameter.
Do the following steps for creating, testing, listing, or deleting a container pair set:
  • To create a container pair set, issue a command similar to this:
    mmcloudgateway containerpairset create --cloud-nodeclass TCTNodeClass1
                                    --container-pair-set-name newContainer
                                    --cloud-service-name myService
                                    --scope-to-filesystem
                                    --path Path 
                                    --data-container DataContainer
                                    --meta-container MetaContainer
    The system displays output similar to this:
    mmcloudgateway: Sending the command to the first successful node starting with v1.gpfs.net
    mmcloudgateway: This may take a while...
    mmcloudgateway: Command completed successfully on vmi.gpfs.net.
    mmcloudgateway: Command completed.
    Note: If you do not specify the names for storing data and metadata containers, then the container pairset name is used for both data and meta containers. In this example, they are "newContainer" (for data) and "newContainer.meta" (for metadata). Start of changeIf you create any meta-containers or data-containers by using any external tools, you can configure containerpairset with these meta-containers or data-containers. To know the bucket creation rules while naming meta-container and data-container for ICOS, S3 and AWS S3 provider, see Rules for Bucket Naming at Bucket restrictions and Limitations.End of change
  • To create a container pairset with thumbnail enabled and the scope is a file system, issue a command similar to this:
    mmcloudgateway containerpairset create --cloud-nodeclass cloud --container-pair-set-name x13
    --cloud-service-name newServ --scope-to-filesystem --path /gpfs --thumbnail-size 64 
  • To create a container pairset when the scope is a fileset, issue a command similar to this:
    mmcloudgateway containerpairset create --cloud-nodeclass cloud --container-pair-set-name x13
    --cloud-service-name newServ --scope-to-fileset --path /gpfs/myfileset
  • To create a container pair set that is enabled for encryption, issue a command similar to this:
    mmcloudgateway containerpairset create --cloud-nodeclass tct --container-pair-set-name
    Containeretag5 --cloud-service-name csss5  --path /gpfs0/fs3 --enc ENABLE --etag ENABLE 
    --data-container test5 --meta-container testmeta5 --key-manager-name lkm3 --scope-to-fileset
    --path /gpfs/myfileset
  • To configure a container pair set using an immutable fileset with a fileset scope, issue a command similar to this:
    mmcloudgateway containerpairset create --cloud-nodeclass tct --container-pair-set-name wormcp
    --cloud-service-name wormservice2 --path /gpfs0/worm2 --enc ENABLE --etag ENABLE --data-container
    wormtestnov --meta-container wormtestnovmeta --key-manager-name lkm3 --scope-to-fileset 
    --path /gpfs/myfileset --cloud-directory-path /gpfs0/fs3
    Note: Here, the fileset is an immutable fileset whereas the cloud directory is pointing to a fileset that is not immutable.
  • To test a container pair set that is created, issue a command similar to this:
    mmcloudgateway containerpairset test --cloud-nodeclass cloud --container-pair-set-name vmip51
    Note: This test will check whether or not the container pair set does actually exist. Additionally, the test will try to add some data to the container (PUT blob), retrieve the data (GET blob), delete the data (DELETE blob), and report the status of each of these operations. This test will validate whether or not all CSAPs for a given container pair set are able to reach the cloud storage.
  • To delete a container pair set, issue a command similar to this:
    mmcloudgateway containerpairset delete --container-pair-set-name x13
    --cloud-nodeclass cloud
  • To list a container pair set, issue a command similar to this:
    mmcloudgateway containerpairset list
    The system displays output similar to this:
    Configured 'containerPairSet' options from node class  cloud:
    ---------------------------------------------------------- 
        containerPairSetName            :  vmip51 
        path                            :  /gpfs/ 
        scopeTo                         :  filesystem 
        transparentRecalls              :  Enabled 
        cloudServiceName                :  newServ 
        dataContainer                   :  vmip51 
        metaContainer                   :  vmip51.meta 
        thumbnailSize                   :  0 
        cloudDirectoryPath              :  /fs/ 
        dataLocation                    :   
        metaLocation                    :   
        activeKey                       :   
        keyManagerName                  :   
        containerPairSetName            :  fset1 
        path                            :  /gpfs/ 
        scopeTo                         :  filesystem 
        transparentRecalls              :  Disabled 
        cloudServiceName                :  newServ 
        dataContainer                   :  fset1 
        metaContainer                   :  fset1.meta 
        thumbnailSize                   :  0 
        cloudDirectoryPath              :  /fs/ 
        dataLocation                    :   
        metaLocation                    :   
        activeKey                       :   
        keyManagerName                  :  
        policyTempDir                   :  /gpfs1
    Configured 'containerPairSet' options from node class cloud2:
    ---------------------------------------------------------- 
        containerPairSetName            :  vmip53 
        path                            :  /x13/ 
        scopeTo                         :  filesystem 
        transparentRecalls              :  Enabled 
        cloudServiceName                :  serv2 
        dataContainer                   :  vmip53 
        metaContainer                   :  vmip53.meta 
        thumbnailSize                   :  0 
        cloudDirectoryPath              :  /x13/ 
        dataLocation                    :   
        metaLocation                    :   
        activeKey                       :   
        keyManagerName                  :  
        policyTempDir                   :  /gpfs1
  • Start of changeTo create a container pair set with auto-spillover disabled, issue a command similar to this:
    mmcloudgateway containerpairset create --cloud-nodeclass tct --container-pair-set-name 
    spilloverdisabled --cloud-service-name csss5 --path /gpfs0/fs3
     --scope-to-fileset --auto-spillover DISABLE
    End of change
  • Start of changeTo create a container pair set with a non-default threshold value for auto-spillover, issue a command similar to this:
     mmcloudgateway containerpairset create --cloud-nodeclass tct 
    --container-pair-set-name spilloverdisabled 
    --cloud-service-name csss5 --path /gpfs0/fs3
    --scope-to-fileset --auto-spillover-threshold <non-default-value>
    To avoid too many spillovers, lower value for threshold is 50 million, whereas upper limit is 100 million entries per container. Non-default value entered in the preceding example, must to be within this range. Run mmcloudgateway containerpairset list to list new container spillover attributes.End of change
Note: Now that you have created your container configuration, it is critical that you do the following:

For more information, see the mmcloudgateway command.

Next step: Backing up the Cloud services configuration