Example of creating an AFM relationship by using the NFS protocol

You can use this example to create an AFM relationship between the home cluster and the cache cluster by using the NFS protocol.

In this example, the fs1 file system is mounted on the /gpfs/fs1 path at the cache cluster and the home cluster. A single-writer (SW) fileset is created at the cache cluster and data is synched to the home fileset and the files are verified. Similarly, other AFM modes fileset can be created by using this example.

  1. Set up the home cluster.

    You can configure the home exports by using either CES NFS or the default NFS, which is available with the operating system.

    1. To create a fileset at a home cluster, run the following command:
       mmcrfileset fs1 fset001 --inode-space new --inode-limit 1000000
    2. To link the fileset, run the following command:
       mmlinkfileset fs1 fset001 -J /gpfs/fs1/fset001
    3. To enable AFM support for extended attributes and sparse files, configure the created fileset.
      1. If you use CES NFS, run the following command to export the path:
         mmnfs export add /gpfs/fs1/fset001 -c "<client Nodes IP/range>(Access_Type=RO,Squash=no_root_squash)"
        For more information about the mmnfs command, see mmnfs command
      2. If you use the default NFS, which is available with the operating system, do the following steps:
        Update /etc/exports and add the following entry:
        /gpfs/fs1/fset001 <client Nodes IP/range>(rw,no_root_squash,no_subtree_check,fsid=101)
        Restart the NFS server.
        # exportfs -ra
        Verify that the added export is shown properly.
        # showmount -e|grep fset001
                /gpfs/fs1/fset001
        Note: The client nodes IP/range must be the gateway node at the cache cluster.
  2. Do the following steps at the cache cluster:
    1. Identify a node and designate the node as a gateway node.
    2. To provide a gateway role to the node in the cache cluster, run the following command:
       mmchnode --gateway -N <Node>
      The system displays a similar output as follows:
      Wed Oct 8 22:35:42 CEST 2019: mmchnode: Processing node <Node>
      	  mmchnode: Propagating the cluster configuration data to all
      	  affected nodes. This is an asynchronous process.
    3. Create an AFM fileset.
       mmcrfileset fs1 fileset_SW -p afmtarget=<home export server>:/gpfs/fs1/fset001 -p afmmode=single-writer --inode-space new
      The system displays a similar output as follows:
      Fileset fileset_SW created with id 1 root inode 131075.
    4. Link the AFM fileset at the cache cluster to sync the AMF relationship with the home cluster.
       mmlinkfileset fs1 fileset_SW -J /gpfs/fs1/fileset_SW
      The system displays a similar output as follows:
      Fileset fileset_SW linked at /gpfs/fs1/fileset_SW
    5. Check whether the state of the cache fileset is inactive.
       mmafmctl fs1 getstate
      The system displays a similar output as follows:
      
      Fileset Name   Fileset Target                Cache State    Gateway Node   Queue Length   Queue numExec
      ------------   --------------                -------------  ------------   ------------   ------------
      fileset_SW     nfs://node4/gpfs/fs1/fset001  Inactive        
      
      Note: Ensure that the home export is mountable on the gateway node at the cache cluster. AFM internally mounts the home NFS export by using NFS v3.
    6. Create a test file 'a' to move the AFM SW fileset from the inactive state to the active state.
      # touch /gpfs/fs1/fileset_SW/a
      # ls -l /gpfs/fs1/fileset_SW/
      The system displays a similar output as follows:
      total 1
      	drwx------ 4 root root 4096 Oct 8 20:38 a

      The AFM fileset state changes to the active state after some time.

      The active state indicates that the home and cache relationship is established and synced.
       mmafmctl fs1 getstate
      The system displays a similar output as follows:
      
      Fileset Name   Fileset Target                Cache State    Gateway Node   Queue Length   Queue numExec
      ------------   --------------                -------------  ------------   ------------   ------------
      fileset_SW     nfs://node4/gpfs/fs1/fset001  Active         node2 	        0              5
      
      Here, 'node2' is the gateway node at the cache cluster and 'node4' is the home export server.
    7. Create more test files in the cache fileset and verify synchronization at the home cluster.
      # cd /gpfs/fs1/fileset_SW
      # for i in 1 2 3 4 ; do date > file$i; done
      1. List the created files in the cache fileset.
         ls -l
        The system displays a similar output as follows:
        total 3 
        	-rw-r--r-- 1 root root 30 Oct 9 20:22 a
        	-rw-r--r-- 1 root root 30 Oct 9 20:25 file1
        	-rw-r--r-- 1 root root 30 Oct 9 20:25 file2
        	-rw-r--r-- 1 root root 30 Oct 9 20:25 file3
        	-rw-r--r-- 1 root root 30 Oct 9 20:25 file4
    8. Check the state of the AFM fileset.
       mmafmctl fs1 getstate
      The system displays a similar output as follows:
      
      Fileset Name   Fileset Target                Cache State    Gateway Node   Queue Length   Queue numExec
      ------------   --------------                -------------  ------------   ------------   ------------
      fileset_SW     nfs://node4/gpfs/fs1/fset001  Dirty           node2 	        8              5
      
    9. Wait for sometime and check again if the fileset state is 'Active'.
       mmafmctl fs1 getstate
      The system displays a similar output as follows:
      
      Fileset Name   Fileset Target                Cache State    Gateway Node   Queue Length   Queue numExec
      ------------   --------------                -------------  ------------   ------------   ------------
      fileset_SW     nfs://node4/gpfs/fs1/fset001  Active         node2 	        0              13
      
  3. To verify the available files in the home cluster, run the following command:
     ls -l /gpfs/fs1/fset001
    The system displays a similar output as follows:
    total 3 
    	-rw-r--r-- 1 root root 30 Oct 9 20:25 a
    	-rw-r--r-- 1 root root 30 Oct 9 20:28 file1
    	-rw-r--r-- 1 root root 30 Oct 9 20:28 file2
    	-rw-r--r-- 1 root root 30 Oct 9 20:28 file3
    	-rw-r--r-- 1 root root 30 Oct 9 20:28 file4