Setting up an active-active GPFS configuration

This example demonstrates how to configure an active-active GPFS cluster.

To establish an active-active GPFS cluster using hardware replication with a tiebreaker site as shown in Figure 1 of An active-active IBM Storage Scale cluster, consider the configuration:

Site A (production site)

Consists of:

Nodes – nodeA001, nodeA002, nodeA003, nodeA004
Storage subsystems – A
Disk volumes – diskA on storage system A
diskA is SAN-attached and accessible from sites A and B

Site B (recovery site)

Consists of:

Nodes – nodeB001, nodeB002, nodeB003, nodeB004
Storage subsystems – B
Disk volumes – diskB on storage system B
diskB is SAN-attached and accessible from site B only

Site C (tiebreaker)

Consists of:

Nodes – nodeC
diskC is an NSD defined over the internal disk of the node nodeC and is directly accessible only from site C

Establish a hardware replication connectivity between the storage systems and then establish the synchronous replication volume pair between the source and target using the copy entire volume option. In this case, it would be diskA–diskB.
In order to protect the order of dependent writes that span multiple disk volumes, multiple storage systems, or both, the consistency group functionality of the storage system should be used with all GPFS devices in the same consistency group.
Create a GPFS cluster defining the primary cluster configuration server as nodes nodeA001 at site A, the secondary cluster configuration server as nodeB001 at site B, an equal number of quorum nodes at each site, including the tiebreaker node at site C, nodeC. To prevent the tiebreaker node from assuming the role of file system manager, define it as client. Define all other quorum nodes as manager. List the nodes in the cluster in the file NodeDescFile. The NodeDescFile file contains the node descriptors:
```
nodeA001:quorum-manager
nodeA002:quorum-manager
nodeA003:quorum-manager
nodeA004:client
nodeB001:quorum-manager
nodeB002:quorum-manager
nodeB003:quorum-manager
nodeB004:client
nodeC:quorum-client
```
Issue this command:
```
mmcrcluster –N NodeDescFile –p nodeA001 –s nodeB001
```
On the tiebreaker node, issue the mmchconfig command to set the unmountOnDiskFail attribute to yes:
```
mmchconfig unmountOnDiskFail=yes -N nodeC
```
This action prevents false disk errors in the SAN configuration from being reported to the file system manager.
Enable the unmountOnDiskFail option on the tiebreaker node preventing false disk errors in the SAN configuration from being reported to the file system manager by issuing the mmchconfig command:
```
mmchconfig unmountOnDiskFail=yes -N nodeC
```
Create an NSD over diskA. The disk descriptor contained in the file DiskDescFile is:
```
/dev/diskA:nodeA001:nodeA002:dataAndMetadata:1
```
Issue this command:
```
mmcrnsd –F DiskDescFileP
```
Start the GPFS daemon on all nodes:
```
mmstartup -a
```
Create a GPFS file system and mount it on all nodes at sites A and B.
```
mmcrfs /gpfs/fs0 fs0 -F DiskDescFile
```