Share load synchronization rules

The basic rule is, only share accounts from global fair share participating clusters, which have the same SAAP (share attribute account path) can sync up share load with each other through gpolicyd. For example, a share account (/ug1/user1) on cluser1 can sync up share load only with share account (/ug1/user1) on cluster2 and share account (/ug1/user1) on cluster3 through gpolicyd.

This rule can be applied to any global share accounts including the share accounts that are created by keywords (default, all, others, group_name@) configured in lsb.users or lsb.queues.

Global fair share policy holds a global fair share tree that is merged from the fair share trees of each participant. The global fair share tree holds global resource usage for share accounts.

The figure below shows the merging process of a global fair share tree. OrgB runs 10 jobs in cluster1 and 8 jobs in cluster2. In the global fair share tree, the node for OrgB holds the global resource usage (18 jobs) for the share account.

Only share accounts with the same share attribute account path (SAAP) from clusters participating in global fair share can synchronize their share loads with each other through gpolicyd.

For example, a share account (/ug1/user1) on cluser1 can only synchronize share load with a share account (/ug1/user1) on cluster2 and a share account (/ug1/user1) on cluster3 through gpolicyd.

Since global fair share is distributed, the fair share tree may be different. In that case, only the nodes with matching SAAPs are updated. The unmatched share load information is dropped. For example, assume clusters A and B participate in one global fair share. All of the nodes need to synchronized their fair share data for global fair share. They have similar fair share trees. Only ug2 has a different configuration:

When user1 submits a job, SAAP for /ug1/user1 will be updated. In the remote cluster, the corresponding SAAP will also be updated

When user3 submits a job, SAAP /ug2/user3 will be updated. In the remote cluster, only the valid corresponding SAAP ug2 will be updated.

Global fair share synchronizes load data when mbatchd is connected to gpolicyd. mbatchd reports its fair share load to gpolicyd every 30 seconds by default. gpolicyd also broadcasts global fair share load to each cluster every 30 seconds by default. You can configure this time interval with the SYNC_INTERVAL parameter in the lsb.globalpolicies configuration file.

Delays are typical for data synchronization in distributed systems: For global fair share, when the local mbatchd receives remote fair share loads, the loads do not reflect the real time loads in other clusters due to any inherent delays and the delay you specified in SYNC_INTERVAL.