Default consistency settings in Global Mailbox

To maximize availability and consistency where required, different default consistency levels are specified for different components in the Global Mailbox system.

Consistency refers to how up-to-date or synchronized data is across all nodes in the system. Consistency is directly related to availability. When the consistency requirement increases more nodes are required and therefore the system can tolerate fewer node failures. Different areas of the system require different levels of consistency. For example, Protocol adapters must write to 50% + 1 nodes in the receiving data center to complete successfully. This ensures that there are sufficient copies of the most up-to-date data within the data center. This also ensures that protocol actions can continue to operate even if another data center is down. The balance between consistency and availability is complex.

Default consistency levels are logged in the messages.logs file. The file is at <install_directory>/wlp/usr/servers/mailboxui/logs. The messages.logs file is created only when you start the Global Mailbox node after installing it. Consistency level for the components is global. A component must have same consistency level in all data center and nodes.
Important: Do not modify the default consistency setting for any of the components. Doing so might adversely impact data integrity, availability, and the overall functioning of the system.

Protocol adapters

Protocol adapters are required for file transfers from and to trading partner users. Default consistency for protocol adapters is LOCAL_QUORUM, which means most of nodes in a specific (local) data center must be available for read and write operations. It does not require quorum across data centers.

When a data center goes down, the global load balancer directs trading partner users to the available data center, implying that the trading partner users interact with only one data center. With the LOCAL_QUORUM consistency level, a trading partner user can complete read and write operations from the available data center.

To ensure data integrity, it is necessary that Cassandra and payload replication use similar consistency levels. If you change payload replication setting, Cassandra consistency setting for protocol adapters is automatically updated to match the replication setting. If payload replication is delayed (asynchronous), consistency setting for protocol adapters is LOCAL QUORUM (default), because interaction with the second data center is not required to complete a transaction. If payload replication is immediate (synchronous), consistency setting is QUORUM, because data must be replicated to at least one other data center to complete the transaction.

The same default consistency applies to uploads and downloads done through myFileGateway.

Global Mailbox Event Rule Adapter

Global Mailbox Event Rule Adapter is required to accept and process Global Mailbox events. To ensure that the adapter does not process a message more than one time and that the processing status is consistent, the default consistency is set to EACH_QUORUM. The consistency level requires read or write done to a quorum of replica nodes in all data centers. With this consistency level, messages are not processed from unplanned data center failures.

Mailbox service

If business processes are run manually, outside the configuration of Global Mailbox Event Rule Adapter, for example, customer scripts, xAPIs, C:D server feature, then the default consistency level is LOCAL_QUORUM. If a business process is configured to run within Global Mailbox Event Rule Adapter, then it is not executed if default consistency for the adapter cannot be met. If payload replication is immediate (synchronous), consistency setting is QUORUM.

Global Mailbox management node and Sterling File Gateway user interface

Default consistency level is LOCAL_QUORUM, which means the data that is shown on the user interface is from the available data center only. During data center failure, there are chances that the data might not be the most recent one, because of the failure of another data center.

The same default consistency applies to Sterling File Gateway administrative activities, such as creating, editing, deleting the partners, template and channel creation that are done from Sterling File Gateway user interface.

Global Mailbox utilities

Default consistency for the following Global Mailbox utilities is LOCAL_QUORUM, because the utilities must tolerate a data center failure, but still be available for configuration changes during the failure:
  • appConfigUtility
  • dbConfigUtility
  • dcConfigUtility
  • schedulerConfigUtility
  • storageConfigurator
  • storagePassphrase

Default consistency for masterPassphrase utility is EACH_QUORUM. To change master passphrase, a quorum across data centers is required.

Scheduled jobs

Default consistency for scheduled jobs is EACH_QUORUM, to protect data from inconsistencies and to ensure purge jobs are deleting correct records. The purge process needs a consistent view before it can be allowed to change the data in any way.

Replicator polling agent

Default consistency is LOCAL_QUORUM. The replicator works with records related to the data center it is running in. It needs a consistent view within the local data center.