Quorum policies supported on Pacemaker

A quorum policy determines how the cluster manager domain will decide quorum to avoid the split brain scenario. Quorum is determined by counting all active nodes within the cluster domain (each cluster node has one quorum vote).

By default, quorum is maintained when more than half the number of nodes ( total_nodes / 2 + 1 ) is available. When a majority of votes is present, cluster operations are allowed to proceed.

When an even number of cluster nodes exist within the cluster domain (excluding pureScale configurations), an additional node, a quorum device node, is needed to act as a tie-breaker during quorum decisions. This allows the cluster to be sustained when half of the cluster nodes are down.

To select an action to take, a cluster manager counts the number of cluster domain nodes that support each of the potential actions. The cluster manager then selects the action that is supported by most of the cluster domain nodes. If the same number of cluster domain nodes supports more than one choice, then the cluster manager refers to a quorum device to make the choice.

The Pacemaker cluster stack supports the following quorum policies:
  • Two-node quorum
  • Quorum device quorum
  • Majority node quorum

Two-node quorum

A two-node quorum is a quorum policy that can be configured such that the cluster can be sustained when one node is down. Because no tie-breaker mechanism exists, the two-node quorum is prone to the split-brain scenario. Due to this, two-node quorum is not supported for production environments within HADR, Mutual Failover, and Data Partitioning Feature (DPF) configurations.

The two-node quorum is only a viable quorum policy for pureScale configurations that contain only two nodes. In the case where a two-node quorum policy is employed, the tie-breaker disk used in the shared filesystem will act as a tie-breaker to avoid the split-brain scenario.

QDevice quorum

QDevice is similar to a Network IP tiebreaker in the sense that it requires an external resource accessible by all hosts in the current Pacemaker cluster. But they differ immensely in terms of reliability. QDevice is much more reliable because the quorum decision logic is more robust than a simple TCP/IP ping to the external IP address. QDevice requires that the resource is placed on a separate host, similar to a majority quorum requirement. However, setup is simplified, as the host with the QDevice does not need to be part of the Pacemaker quorum. The QDevice is the best blended quorum solution that combines reliability and simplicity.

Figure 1. An overview of the key components in a Pacemaker cluster that uses QDevice quorum
An overview of the key components in a Pacemaker cluster that uses QDevice quorum
Consider the following information about QDevices:

  • A quorum device acts as the third-party arbitration device for the cluster. Its primary use is to allow a cluster to sustain more node failures than standard quorum rules allow.

  • As seen in Figure 1, it is a Corosync daemon process, QDevice daemon (corosync-qdevice), running on each node in the cluster. The QDevice daemon provides a configured number of votes to the quorum subsystem, based on a third-party arbitrator's decision. This third-party arbitrator is a separate Corosync QNet daemon (corosync-qnetd) running on a separate host (not part of the cluster). The third-party arbitrator contributes to the deciding vote of the corosync-qdevice logic that ultimately decides the surviving side in a split-brain scenario.

  • Both the QDevice daemon and the QNet daemon are provided with different software packages and must be installed separately. The QDevice daemon must be installed on each host in the cluster (Host1 and Host2 in Figure 1). The QNet daemon is needed only on a separate host that is not part of the cluster (Host3 in Figure 1).

  • The QNet daemon can be used as the arbitrator for another cluster (in Figure 1, two clusters share the same host with qnetd process), given that all clusters have a unique name.

  • The arbitrator host running the QNet daemon does not need to be running the same operating system, or hardware architecture, as the Pacemaker cluster running the QDevice daemon.
  • QDevice quorum is required if the cluster contains an even number of nodes.
Figure 2. A two-node configuration with Qdevice configured.
A two-node HADR configuration with Qdevice configured
Table 1. Scenarios and expected behaviors in a two-node configuration with Qdevice configured.
Link Failures Host1 instance state Host1 Quorum status Host2 instance state Host 2 Quroum status Host3 state Scenario description
1 Up Acq Down Lost Up Quorum device picks the node with the lowest node ID value to survive, which is Host 1. Host 2 loses quorum and the instance is taken down.
2 Up Acq Up Acq Up Host 1 still has quorum and continues to work.
3 Up Acq Up Acq Up Host 2 still has quorum and continues to work.
1 and 2 Down Acq Up Acq Up Host 1 loses quorum and the instance is taken down.
2 and 3 Up Acq Up Acq Down Host 1 and host 2 maintain quorum but are without the quorum service.
1 and 3 Up Acq Down Lost Up Host 2 loses quorum and the instance is taken down.
All Down Lost Down Lost Down Both hosts lose quorum and the instances are taken down. The quorum service is also down.
For information on the requirements of the third arbitrator host, see Prerequisites for an integrated solution using Pacemaker.

Majority quorum

The Majority Quorum can be used when an odd number of nodes are integrated within the cluster. In a split-brain scenario, the side that successfully acquires the last node (odd node) is the surviving side. Unlike a QDevice node, the odd node must be fully integrated within the cluster. Majority Quorum is not supported for HADR and Mutual Failover configurations. For Data Partitioning Feature (DPF) deployments, majority quorum is the default quorum mechanism

SBD fencing

For Mutual Failover clustered environments, SBD fencing is configured along with Qdevice. If a quorum loss occurs, the host with a quorum loss is fenced (rebooted) by the watchdog that is configured in /dev/watchdog. If it is not already configured on your system, the /dev/watchdog is configured as part of your Db2 installation.
Note: Qdevice is required for production-level Mutual Failover environments.

Quorum consideration on public cloud vendors

For more information, refer to Public cloud vendors supported with Db2 Pacemaker.

Table 2. The advantages and disadvantages of each quorum type
Quorum type Advantages Disadvantages
Two Node
  • Simplest setup.
  • No additional hardware or software configuration.
  • Potential split-brain scenario, leading to dual primary phenomenon.
QDevice
  • More reliable than Two-Node quorum.
  • No need to include the third host as part of the cluster.
  • No need to include the full Pacemaker cluster software stack on the third host. Only one Corosync RPM is needed.
  • Requires a TCP/IP accessible host from the primary and standby hosts.
Majority
  • More reliable than Two-Node quorum.
  • Need to include the full Pacemaker cluster stack that is installed and configured on the third host.