About changing resource distribution models

You change your current resource distribution model (from either full ownership with lending and borrowing or full share with no ownership) to a hybrid model that supports both resource ownership and sharing.

Scenario A: From Ownership to Hybrid model

As a cluster administrator, you originally set up a resource distribution model that supports full resource ownership. You may have enabled lending and borrowing of resources within the cluster, or you may have set up a silo model where there is no borrowing or lending of resources between consumers.

You now wish to change models so that consumers can have access to both owned resources (either their own or those allocated to other consumers) and shared cluster resources (unowned resources that make up the cluster’s share pool). You want a more flexible and less prescribed model of resource distribution that responds better to fluctuating workload conditions and number of cluster resources.

Scenario B: From Share to Hybrid model

As a cluster administrator, you set up a distributed resource sharing model where there are no owned slots, but where each consumer has an assigned share ratio of cluster resources (share pool).

Your company has a new line of business that requires specific resources (for example, from a certain resource group). The previous full share model, where resources were flexibly allocated depending on workload conditions, consumer priority, and resource availability, cannot guarantee the allocation of the hosts required to satisfy the needs of the new line of business. You want to implement a hybrid model where some resources remain in the share pool, while others are allocated specifically to the new line of business in order to guarantee a minimum level of resource allocation.

At a glance

  1. Gather the facts
  2. Changing the resource plan to support a Hybrid model
  3. Adding a new consumer to a Hybrid model resource plan
  4. Viewing the number of owned and shared slots

Gather the facts

EGO supports three resource distribution models in the resource plan.
Table 1. Resource distribution models
Ownership model (“silo”, or with lending/borrowing) Share model Hybrid model
In this model, resource distribution ranges from “silo” (where each consumer has a fixed ownership number with no lending and borrowing between them), to limited lending/borrowing among consumers, to unlimited lending and borrowing of resources between consumers.
Note: Share limits are set to zero for all consumers in this model.
 In this model, each consumer has a portion (assigned share ratio) of the clusters resources (“share pool”). Each consumer has the potential to use resources from the share pool if other consumers have no competing demand for resources. If there is competition between consumers for resources from the share pool, note the following:
  1. A consumer’s rank is enforced. A consumer with a higher rank than its sibling receives its ratio of share pool resources first.
  2. A consumer’s assigned share ratio is enforced. EGO automatically reclaims the resources from the over-allocated consumer.
Note: Consumers do not own any slots in this model. Set ownership to 0, as lending and borrowing configurations are of no effect. Ensure share ratios are suitably configured across all consumer branches and for each consumer.
 In this model, you can create a resource plan that combines both models (Hybrid model). In this model, EGO maintains a share pool and also supports configured resource ownership by leaf consumers.
The following diagram shows all resource distribution models and potential model migrations. Each arrow represents either a complete migration from one model to another (there are six migration possibilities) or a change within a current resource distribution model.
Note: The dotted arrows indicate those resource distribution migrations or changes with notable restrictions. Resource distribution model migrations or changes indicated by solid arrows have no restrictions or associated issues.
All resource distribution models and potential model migrations. Each arrow represents either a complete migration from one model to another (there are six migration possibilities) or a change within a current resource distribution model. dotted arrows indicate those resource distribution migrations or changes with notable restrictions. Resource distribution model migrations or changes indicated by solid arrows have no restrictions or associated issues.

There are restrictions for the resource distribution model migrations and changes indicated by dotted arrows.

Restrictions when migrating from a Share to a Hybrid model

When you migrate from a Share model to a Hybrid model, resources that were previously in the share pool are allocated as owned resources to consumers in the resource plan; the number of share pool resources therefore decreases, which may potentially cause difficulties.

In some cases, you may not be able to immediately satisfy the ownership requirements set out in your resource plan when migrating from a Share to a Hybrid model. When the new resource plan comes into effect, and the unallocated share pool resources (in the original Share model) are fewer in number than the configured ownership number in the plan (in the new Hybrid model), then the resource distribution requirements of the plan cannot be fulfilled.

Example: In the Share model, the share pool has 100 slots: 60 slots (resources) are already allocated to consumers running workload units, and 40 slots are available for allocation. Upon implementing a Hybrid model, you intend to reduce the share pool to 50 slots and distribute the remaining 50 slots to three consumers (consumers A, B, and C) with an ownership allocation of 20, 20, and 10 respectively. But after reconfiguring the resource plan, consumer C cannot get its 10-slot allocation. This occurs because there are resources in the share pool that are already allocated. During a resource plan migration from a Share to a Hybrid model, the normal logic for reclaiming share pool resources is not enforced.

Summary: If there are not enough unallocated resources in the share pool at the moment the new Hybrid plan takes effect (during runtime), then consumers are not allocated the planned number of owned resources. Allocated resources from the share pool must first be released back to the share pool by the clustered application managers that are using them (for example, IBM® Spectrum Symphony or IBM Spectrum LSF®) before they can be reallocated as owned resources to a consumer.

Restrictions when making resource plan changes within a Hybrid model

When modifying a resource plan for an existing Hybrid model (where there are both owned resources and unowned share pool resources), some issues may arise.

  • If you add a new leaf consumer and allocate resources to it by taking them from the share pool, the consumer’s guaranteed ownership allocation may not be satisfied if the share pool resources are in use by other consumers. If there are not enough unallocated share pool resources to fulfill the ownership requirements in the new resource plan, then the new consumer does not receive its planned resources until sufficient resources are released back into the share pool.
  • If you allocate share pool resources as owned resources to a consumer at any level in the consumer tree (first-level, parent, or child), and there are insufficient unallocated share pool resources to fulfill the ownership requirements of the new resource plan, consumers throughout the branch remain unsatisfied. First-level consumers are satisfied first with available resources, according to the plan, followed by parents and their children.

There are no issues if resources taken from the share pool are available (if they are not currently allocated to other consumers).

These use-cases do not have any restrictions or associated issues:
  • Adding a new consumer with ownership without changing the number of resources owned by the parent.
  • Reducing the number of resources owned by a consumer and increasing the share pool or changing the share ratio.
  • Removing some consumers.

Changing the resource plan to support a Hybrid model

You already have a working resource plan that supports either a Share or Ownership model. You may want to export the existing resource plan and save it for later re-use before making your changes.

When implementing a Hybrid resource plan, ensure that there are enough unallocated resources in the share pool at the time that the new resource plan becomes effective. Unless specified, each step in the following task relates to migrations from either Share or Ownership models.

  1. Log in to the cluster management console and go to Resources > Resource Planning (Slot) > Resource Plan.
  2. (Original mode type: Share) To reclaim allocated share pool resources, and thereby increase the number of resources available for ownership in the plan, set the Limit in the Model Type: Share section to 0 for all consumers.

    This forces the allocated resources to immediately return to the share pool. You can then allocate reclaimed resources from share pool to a consumer.

    If you take this approach, your workload units may be disrupted if it is using the share pool resources that are reclaimed. Create a new time interval to avoid this issue:
    1. In the Resource Plan, create at least one new time interval. You want a time interval for the existing Share model resource plan, an interval for the new Hybrid model resource plan, and a short interval (for example, 1 hour) to fall between the Share and Hybrid model plans.

      The short time interval provides a buffer between the two resource plans, and allows currently allocated share pool resources that are running workload units to be released back to the share pool before the Hybrid plan becomes effective. The short time interval must be longer than the grace period for resource reclaim.

    2. In the short time interval that falls between the Share and Hybrid plans, set the Limit in the Model Type: Share section to 0 for all consumers.
    3. In the time interval that follows, set the desired resource policies to support a Hybrid model of distribution.
  3. Set or adjust owned slots for each consumer to guarantee a minimum allocation of resources.
    When all available resources are allocated, the cluster balance in the Owned Slots section changes to 0.
    Important: If you are allocating resources from the share pool to a consumer, ensure there are enough unallocated resources in the share pool at the time that the new resource plan becomes effective to satisfy the consumer’s configured ownership.
  4. Set or adjust a lend policy for each consumer; enable lending for leaf consumers who own resources.
  5. Set or adjust a borrowing policy for each consumer; enable borrowing for leaf consumers who own resources.
  6. Set or adjust share ratios for each consumer.
  7. Set or adjust the consumer rank for each consumer.
  8. Click Apply when all changes the resource plan are complete and you are ready to immediately implement the new plan.

Adding a new consumer to a Hybrid model resource plan

You already have a working resource plan that supports a Hybrid model.
  1. Create a new consumer:
    1. From Resources > Consumers, navigate to the location in your consumer tree where you wish to add the new consumer, and then select Create a Consumer from the Global Actions list.
    2. Fill in the consumer properties.
  2. Update the resource plan:
    1. Go to Resources > Resource Planning (Slot) > Resource Plan.
    2. Set or adjust owned slots for each consumer to guarantee a minimum allocation of resources.
      When all available resources are allocated, the cluster balance in the Owned Slots section changes to 0.
      Important: If you are allocating resources from the share pool to the new consumer, ensure there are enough unallocated resources from the share pool at the time that the new resource plan becomes effective. Otherwise, the new consumer does not receive its configured ownership until sufficient resources are released back into the share pool.
    3. Set or adjust a lend policy for each consumer; enable lending for leaf consumers that own resources.
    4. Set or adjust a borrowing policy for each consumer; enable borrowing for leaf consumers that own resources.
    5. Set or adjust share ratios for each consumer.
    6. Set or adjust the consumer rank for each consumer.
  3. Click Apply when all changes the resource plan are complete and you are ready to immediately implement the new plan.

Viewing the number of owned and shared slots

For comparison purposes, you can view the number shared slots that a consumer gets from the share pool along with the number of owned slots. Do this using the cluster management console or using the command line.
  • From the cluster management console, click Resources > Resource Planning (Slot) > Resource Allocation.
  • From the command line, run the egosh consumer view consumer_name command.