Example of creating an AFM relationship by using the NFS protocol
You can use this example to create an AFM relationship between the home cluster and the cache cluster by using the NFS protocol.
In this example, the fs1 file system is mounted on the /gpfs/fs1 path at the cache cluster and the home cluster. A single-writer (SW) fileset is created at the cache cluster and data is synched to the home fileset and the files are verified. Similarly, other AFM modes fileset can be created by using this example.
- Set up the home cluster.
You can configure the home exports by using either CES NFS or the default NFS, which is available with the operating system.
- To create a fileset at a home cluster, run the following
command:
mmcrfileset fs1 fset001 --inode-space new --inode-limit 1000000
- To link the fileset, run the following
command:
mmlinkfileset fs1 fset001 -J /gpfs/fs1/fset001
- To enable AFM support for extended attributes and sparse files, configure the created fileset.
- If you use CES NFS, run the following command to export the
path:
For more information about the mmnfs command, see mmnfs commandmmnfs export add /gpfs/fs1/fset001 -c "<client Nodes IP/range>(Access_Type=RO,Squash=no_root_squash)"
- If you use the default NFS, which is available with the operating system, do the following
steps:Update /etc/exports and add the following entry:
Restart the NFS server./gpfs/fs1/fset001 <client Nodes IP/range>(rw,no_root_squash,no_subtree_check,fsid=101)
Verify that the added export is shown properly.# exportfs -ra
# showmount -e|grep fset001 /gpfs/fs1/fset001
Note: The client nodesIP/range
must be the gateway node at the cache cluster.
- If you use CES NFS, run the following command to export the
path:
- To create a fileset at a home cluster, run the following
command:
- Do the following steps at the cache cluster:
- Identify a node and designate the node as a gateway node.
- To provide a gateway role to the node in the cache cluster, run the following
command:
The system displays a similar output as follows:mmchnode --gateway -N <Node>
Wed Oct 8 22:35:42 CEST 2019: mmchnode: Processing node <Node> mmchnode: Propagating the cluster configuration data to all affected nodes. This is an asynchronous process.
- Create an AFM
fileset.
The system displays a similar output as follows:mmcrfileset fs1 fileset_SW -p afmtarget=<home export server>:/gpfs/fs1/fset001 -p afmmode=single-writer --inode-space new
Fileset fileset_SW created with id 1 root inode 131075.
- Link the AFM fileset at the cache cluster to sync the AMF relationship with the home
cluster.
The system displays a similar output as follows:mmlinkfileset fs1 fileset_SW -J /gpfs/fs1/fileset_SW
Fileset fileset_SW linked at /gpfs/fs1/fileset_SW
- Check whether the state of the cache fileset is
inactive.
The system displays a similar output as follows:mmafmctl fs1 getstate
Fileset Name Fileset Target Cache State Gateway Node Queue Length Queue numExec ------------ -------------- ------------- ------------ ------------ ------------ fileset_SW nfs://node4/gpfs/fs1/fset001 Inactive
Note: Ensure that the home export is mountable on the gateway node at the cache cluster. AFM internally mounts the home NFS export by using NFS v3. - Create a test file 'a' to move the AFM SW fileset from the inactive state to the active
state.
The system displays a similar output as follows:# touch /gpfs/fs1/fileset_SW/a # ls -l /gpfs/fs1/fileset_SW/
total 1 drwx------ 4 root root 4096 Oct 8 20:38 a
The AFM fileset state changes to the active state after some time.
The active state indicates that the home and cache relationship is established and synced.
The system displays a similar output as follows:mmafmctl fs1 getstate
Here, 'node2' is the gateway node at the cache cluster and 'node4' is the home export server.Fileset Name Fileset Target Cache State Gateway Node Queue Length Queue numExec ------------ -------------- ------------- ------------ ------------ ------------ fileset_SW nfs://node4/gpfs/fs1/fset001 Active node2 0 5
- Create more test files in the cache fileset and verify synchronization at the home cluster.
# cd /gpfs/fs1/fileset_SW # for i in 1 2 3 4 ; do date > file$i; done
- List the created files in the cache fileset.
The system displays a similar output as follows:ls -l
total 3 -rw-r--r-- 1 root root 30 Oct 9 20:22 a -rw-r--r-- 1 root root 30 Oct 9 20:25 file1 -rw-r--r-- 1 root root 30 Oct 9 20:25 file2 -rw-r--r-- 1 root root 30 Oct 9 20:25 file3 -rw-r--r-- 1 root root 30 Oct 9 20:25 file4
- List the created files in the cache fileset.
- Check the state of the AFM fileset.
The system displays a similar output as follows:mmafmctl fs1 getstate
Fileset Name Fileset Target Cache State Gateway Node Queue Length Queue numExec ------------ -------------- ------------- ------------ ------------ ------------ fileset_SW nfs://node4/gpfs/fs1/fset001 Dirty node2 8 5
- Wait for sometime and check again if the fileset state is
'Active'.
The system displays a similar output as follows:mmafmctl fs1 getstate
Fileset Name Fileset Target Cache State Gateway Node Queue Length Queue numExec ------------ -------------- ------------- ------------ ------------ ------------ fileset_SW nfs://node4/gpfs/fs1/fset001 Active node2 0 13
- To verify the available files in the home cluster, run the following
command:
The system displays a similar output as follows:ls -l /gpfs/fs1/fset001
total 3 -rw-r--r-- 1 root root 30 Oct 9 20:25 a -rw-r--r-- 1 root root 30 Oct 9 20:28 file1 -rw-r--r-- 1 root root 30 Oct 9 20:28 file2 -rw-r--r-- 1 root root 30 Oct 9 20:28 file3 -rw-r--r-- 1 root root 30 Oct 9 20:28 file4