Creating and managing hybrid placement rules

You can create and manage hybrid placement rules for hybrid applications to define where and how hybrid application components, such as hybrid deployables, deployables, and Helm charts are deployed. Use hybrid placement rules to help you facilitate multi-cluster deployments of your deployables.

Hybrid and non-hybrid placement rules

Within IBM Cloud Pak for Multicloud Management there are two types of placement rules, one for deploying pure Kubernetes-based application resources and the second, hybrid placement rules, for deploying hybrid application resources. If your application resource is a hybrid deployable, use a hybrid placement rule for defining where to place the resource.

The pure Kubernetes-based placement rules (PlacementRule.mcm.ibm.com) define the cluster destination where application resources are to be placed. For more information about placement rules for deploying only pure Kubernetes resources, see Creating and managing placement rules.

Hybrid placement rules (PlacementRule.core.hybridapp.io) are a custom enhancement of the pure Kubernetes-based placement rule, which enables the rule to be used for deploying more than pure Kubernetes resources. These hybrid placement rules use a placement by deployer approach for placing hybrid application resources on target destinations.

With hybrid placement rules, you can place resources on clusters and other targets that are outside of the default cluster definition. For instance, you can place hybrid deployable resources on different deployer types, such as virtual machines, cloud services, and containers. These deployers can exist on the hub cluster or on a managed cluster. For more information about deployers, see Creating and managing deployers.

Hybrid placement rules also include a mechanism for integrating with third-party advisors for recommendations on where to place hybrid application resources.

Notes:

If you have existing hybrid deployables that reference pure-kubernetes placement rules, those rules continue to work, however you are recommended to update your hybrid deployables to use hybrid placement rules.
Subscriptions (Subscription.app.ibm.com) do not support hybrid placement rules. Unlike placement rules for pure Kubernetes-based deployables, which can be defined for subscriptions, hybrid placement rules are defined specifically for deployables whether the deployable is for a single or multi-cluster deployment.

Placement recommendations and decisions

To support deployments to other deployer types, hybrid placement rules introduce a decision-making process that enables the Hybrid Application Model to work with advisors to determine the ideal target where to place resources. An advisor can be an external controller, such as a deployer or third-party deployer, that provides recommendations to the placement rule for deciding where to place a deployable. These recommendations can be a target deployer or cluster. For this decision-making process, the status section of a hybrid placement rule is used to list the potential candidate destinations, identify recommendations, and list the resulting decisions, or actual target destinations.

The deployerType is used to populate the list of potential target candidates in a status.candidates section. These candidates are then intercepted by any specified advisors. The advisors provide the recommended targets for the candidates and send updates for the placement rule status to list the recommendations in a status.recommendations section. The number of recommendations can exceed the defined spec.replicas value. The replicas value only limits the number of targets that are included within the status.decisions section that lists the resulting target destinations. Instead, the status.recommendations section lists all recommendations that are returned by the advisors. The hybrid placement rule spec can include rules for the recommendations, such as labels that must exist on the recommended targets. You can also define weights for the advisors to prioritize recommendations from specific advisors.

Note:

If advisors are specified and provide recommendations for the specified candidates, any potential target destination that is defined outside of the listed candidate recommendations is either added to the list of candidates. For instance, if a deployer or cluster is specified as the target elsewhere in the placement rule, that specified deployer or cluster is added as a candidate.
If no advisor is specified, the hybrid placement rule is used to directly determine placement, such as from a specified target deployer or cluster or to the default cluster namespace on the hub cluster.

The list of final target decisions is populated within the status.decisions section after all candidate recommendations are evaluated by the placement rule controller. The decisions are an array of objects. Each object is described by a kind, a name, and a namespace. The placement rule controller evaluates the recommendations by using the specified rules, advisor weights, and the recommendation scores. The status.decisions section lists the selected recommendations. To evaluate the recommendations, the placement rule controller goes through a series of reconciallition cycles.

First, the placement rule spec is checked for any defined target deployers or clusters. If any such targets are found, they are added as candidates. After the list of candidates is populated with the defined candidates and any defined targets, the advisors provide recommendations for the candidates based on any defined rules.

If the initial list of recommendations matches or is less than the spec.replicas value, the recommendations are all listed in the decisions section. If the number of recommendations exceed the spec.replicas value, the placement rule controller calculates a rating for each candidate and removes the lowest rating candidate. The advisors are then used to provide recommendations for the reduced candidate list. This process is repeated until the list of recommendations matches or is less than the spec.replicas value.

The rating for candidates is calculated by multiplying the score for each recommendation by the weight for the associated advisor and then combining the values for each candidate.

For example, candidate A is provided two recommendations:

Recommendation 1, which includes a score of 20 (20/100) and is provided by advisor A, which has a weight of 60. This recommendation value is 12.
Recommendation 2, which includes a score of 80 (80/100) and is provided by advisor B, which has a weight of 50. This recommendation value is 40.

The candidate rating is the 52.

Example evaluation

The following placement rule includes a list of potential candidates. In addition, a previous evaluation determined cluster3 to be the selected target destination:

apiVersion: core.hybridapp.io/v1alpha1
kind: PlacementRule
metadata:
  name:
spec:
  deployerType:
  replicas: 1
  targets:
  - name:
  targetLabels:
    matchLabels:
  decisionWeight:
  advisors:
  - name: grc
    type: predicate
    rules:
      policies:
      - name:
  - name: rhacm
    type: priority
    weight: 60
    rules:
      resourceHint:
        type: cpu
  - name: cost
    type: priority
    weight: 50
status:
  lastUpdateTime:
  candidates:
  - name: cluster1
  - name: cluster2
  - name: cluster3
  decisions:
  - name: cluster3

After the advisors receive the candidates, the advisors provide recommendations for the candidates. This list of recommendations exceeds the status.replicas value so the list of candidates needs to be reduced and new recommendations provided. As such no changes to the list of decisions occur:

status:
  lastUpdateTime:
  candidates:
  - name: cluster1
  - name: cluster2
  - name: cluster3
  recommendations:
    rhacm:
    - name: cluster3
    grc:
    - name: cluster1
    - name: cluster3
    cost:
    - name: cluster2
  decisions:
  - name: cluster1

As the advisor grc is a predicate type advisor, a recommendation from this advisor must be selected as the decision. Either cluster1 or cluster3 must be selected. To determine the candidate to remove the following ratings are calculated:

cluster1: (100/100) * 60 = 60
cluster2: (100/100) * 50 = 50
cluster3: ((100/100) 60) + ((100/100) 100) = 160

No score is provided for either advisor so the default score 100 (100/100) is used. As no weight is set for the grc advisor, the default weight 100 is used. The candidate cluster2 has the lowest rating and is removed. With this removal, the advisors provide new recommendations:

status:
  lastUpdateTime:
  candidates:
  - name: cluster1
  - name: cluster3
  recommendations:
    rhacm:
    - name: cluster3
    grc:
    - name: cluster1
    - name: cluster3
    cost:
    - name: cluster1
  decisions:
  - name: cluster1

The spec.replicas value is still not matched, so the candidate ratings are recalculated to reduce the list again:

cluster1: ((100/100) 60) + ((100/100) 50) = 110
cluster3: ((100/100) 60) + ((100/100) 100) = 160

The candidate cluster1 has the lowest rating and is removed. With this removal, the advisors provide new recommendations:

status:
  lastUpdateTime:
  candidates:
  - name: cluster1
  recommendations:
    rhacm:
    - name: cluster1
    grc:
    - name: cluster1
    cost:
    - name: cluster1
  decisions:
  - name: cluster1

The spec.replicas value is now matched, so the candidate recommendation cluster1 is selected as the decision for the target destination.

Creating a placement rule

If you choose to create an application by assembling resources and create hybrid deployables in the process, a hybrid placement rule is created automatically for each deployable. Each hybrid deployable includes the placementRef setting to identify the placement rule. For more information about application assembly, see Creating and managing application assemblers.

To manually create a placement rule, you can use the Kubernetes CLI (kubectl) tool or API:

Compose the definition YAML content for your placement rule. For more information about the YAML structure, including the required fields, see Placement rule definition.
Create the placement rule within IBM Cloud Pak for Multicloud Management:
- To use the Kubernetes CLI tool:
  1. Run the following command to apply your file to an apiserver. Replace filename with the name of your file:
```
kubectl apply -f filename.yaml
```
  2. Verify that your placement rule is created, by running the following command:
```
kubectl get PlacementRule
```
    Ensure that your new placement rule is listed in the resulting output.
- To use REST API, you need to use the hybrid placement rule POST API.
Assign the placement rule.

Assigning a placement rule

Hybrid deployables are updated to support including references to hybrid placement rules instead of specifically identifying a Deployer resource, such as for representing infrastructure deployers. The referenced hybrid placement rule defines the deployerType and other settings for determining where to place the hybrid deployable.

You can assign a placement rule to a hybrid deployable. To assign a placement rule, you need to update the spec for the hybrid deployable to reference the placement rule. For more information about creating or updating the definition for a hybrid deployable, see Creating and managing hybrid deployables

Include the following placement fields in the spec with the values to reference the hybrid placement rule:

placement:
  placementRef:
    name:

Include the name of your hybrid placement rule as the value for the name field.

Viewing the status of a placement rule

When a hybrid placement rule is created and in use, you can view the status details for the rule. This status is appended to the YAML definition for a placement rule and indicates the decisions for selecting the target destinations where hybrid deployables are to be placed. For more information about the status details for a rule, see Placement rule status definition.

To view the status fields for a placement rule, you can use the console, the Kubernetes command line interface (kubectl) tool, or REST API.

To use the console,
1. Open the console.
2. Click the Search icon in the Header.
3. Within the search box, filter by kind:placementrule to view all placement rules.
4. Within the list of all placement rules, click the placement rule that you want review. The YAML for that rule is displayed.
5. Review the fields and values within the status section of the YAML content.
To use the Kubernetes CLI tool, run the following command. Replace name and namespace with the name of the placement rule and the target namespace:
1. Run the following command
```
kubectl get PlacementRule <name> -n <namespace>
```
2. Review the fields and values within the status section of the YAML content.
To use REST API, you need to use the hybrid placement rule GET API.

Updating a placement rule

To update a placement rule, you can use the console, the Kubernetes command line interface (kubectl) tool, or REST API.

To use the console to edit a placement rule, complete the following steps:
1. Open the console.
2. Click the Search icon in the Header.
3. Within the search box, filter by kind:placementrule to view all placement rules.
4. Within the list of all placement rules, click the placement rule that you want to update. The YAML for the rule is displayed.
5. Click Edit to enable editing the YAML content.
6. When you are finished your edits, click Save. Your changes are saved and applied automatically.
To use the Kubernetes CLI tool, the steps are the same as for creating a placement rule.
To use REST API, you need to use the hybrid placement rule PATCH API.

Deleting a placement rule

Before you delete a rule, ensure that any hybrid deployable that references the rule is updated to reference a different placement rule.

To delete a hybrid placement rule, you can use the console, the Kubernetes command line interface (kubectl) tool, or REST API.

To use the console, use the console search to find and delete a placement rule:
1. Open the console.
2. Click the Search icon in the Header.
3. Within the search box, filter by kind:placementrule to view all placement rules.
4. Within the list of all placement rules, expand the Options menu for the placement rule that you want to delete. Click Delete placement rules.
5. When the list of all placement rules is refreshed, the placement rule is no longer displayed.
To use the Kubernetes CLI tool, complete the following steps:
1. Run the following command to delete the placement rule from a target namespace. Replace name and namespace with the name of your placement rule and your target namespace:
```
kubectl delete PlacementRule <name> -n <namespace>
```
2. Verify that your placement rule resource is deleted by running the following command:
```
kubectl get PlacementRule <name>
```
To use REST API, you need to use the hybrid placement rule DELETE API.

Placement rule definition YAML structure

The following YAML structure shows the required fields for a hybrid placement rule and some of the common optional fields. Your YAML structure needs to include some required fields and values. Depending on your application management requirements, you might need to include other optional fields and values. You can compose your own YAML content with any tool.

apiVersion: core.hybridapp.io/v1alpha1
kind: PlacementRule
metadata:
  name:
spec:
  deployerType:
  replicas:
  targets:
  - name:
  targetLabels:
    matchLabels:
  decisionWeight:
  advisors:
  - name:
    type:
    rules:
      policies:
      - name:
  - name:
    type:
    weight:
    rules:
      resourceHint:
        type:
  - name:
    type:
    weight:

Table 1. Required and optional definition fields
Field	Required	Description
`apiVersion`	Y	Set the value to `core.hybridapp.io/v1alpha1` for defining where to place hybrid application resources.
`kind`	Y	Set the value to `PlacementRule` to indicate that the resource is a placement rule.
`metadata.name`	Y	The name for identifying the placement rule.
`deployerType`	N	A string that defines the type of deployer. The deployer type for a placement rule can be defined as all types or as an array of allowed types. The default setting is `kubernetes`, which indicates that the deployed resources are pure Kubernetes resources. If a deployer is associated with a virtual machine hybrid deployable, set the `deployerType` to be `infrastructure`. The target destination where to place deployables for a deployer is defined within the resource definition of the deployer.
`replicas`		A UInt16 value that defines the maximum number of target destinations (candidates) to be selected from the deployers to create the list of decisions.
`targets`		An array of `v1.ObjectReference` for directly specifying target candidates for where to place deployables.
`targetLabels`		A label selector (`metav1.LabelSelector`) for identifying target candidates. This label selector is used in combination (`AND` relationship) with the `targets` field when both fields are populated.
`advisors`		An array introduced by the Hybrid Application Model for identifying advisors. An advisor is identified by its name, which is watched by the advisor controller.
`advisor.type`	N	Defines the type of an advisor. The type can be `predicate` or `priority`. The default type is `priority`. A predicate advisor indicates that the advisor can veto target candidates that are not included in its recommendations so that any recommendation from the advisor must be included in the list of decisions and used. The recommended behavior of a predicate advisor is to provide as many candidates as possible beyond the defined `spec.replicas` setting to improve the chances of deciding where to place deployables. The hybrid placement rule operator is a predicate advisor. The recommendations from the operator are included within the `status.candidates` setting.
`advisor.weight`	N	Defines the weight of the recommendations that are provided by the advisor. The higher the weight, the more likely the recommendations are used. The default weight is 100. Within the decision-making cycle, the hybrid placement rule operator measures the weight from all advisors for decision making.
`advisor.rules`	N	Defines specific criteria for each advisor. The rules are handled for each advisor individually. If the advisor always provides a recommendation by using the same rules, this field can be empty. The `status.candidates` setting that is populated by the placement rule can show the recommendations from the rules and the `deployertype`, `targets`, and `targetLabels` settings.
`decisionWeight`		Defines the weight of existing decision in a recalculation of placement decisions. Different workloads might set it differently.

With the hybrid application model, you can define the deployer and deployer namespace for placing resources. In addition, you can define a target cluster name and cluster namespace. Only set values for determining the target cluster or the target deployer, not both. When you are defining the target clusters or deployers, ensure that the target can recognize and process the deployable or hybrid deployable. For example, you cannot deploy a Helm chart to Infrastructure management. The defined spec.capabilities for a deployer resource defines what kind of resources that deployer can process.

Status definition YAML structure

Created hybrid placement rules can include the following fields to indicate the status for the hybrid placement rule. This status section is appended after the spec section in the YAML structure.

The following YAML content shows the structure of the status section of a hybrid placement rule:

status:
  lastUpdateTime:
  candidates:
  - name:
  recommendations:
    advisors:
    - name:
      score:
    cost:
    - name:
      score:
    rhacm:
    - name:
    grc:
    - name:
  decisions:
  - name:

Table 2. Status definition fields
Field	Description
`status`	The status information for the placement rule.
`lastUpdateTime`	The last time that the resource was updated. This time can be used to time out the decision-making process and force a decision for where to place deployables.
`candidates`	This field is populated by the hybrid placement rule operator to initially filter targets for where to place deployables.
`recommendations`	A map that is populated by advisors. They key of the map is the advisor name.
`advisors`	The map that includes recommendations from the advisors. Each advisor provides a list or object that has an optional associated score.
`advisors.score`	An integer in the range 0 - 100. The default score is 100. This value is used to populate the status.decisions settings by the hybrid placement rule operator to determine where to place deployables.
`decisions`	Defines the target clusters where deployables are placed. The hybrid deployable operator checks the `status.decisions` value to determine the actual deployment actions to complete.
`decisions.name`	The name of a target cluster.
`decisions.namespace`	The namespace for a target cluster.

Example 1: Cluster placement

The following example status.decisions section includes a decision setting that indicates resources should be placed on the specified cluster.

status:
  candidates:
  - apiVersion: clusterregistry.k8s.io/v1alpha1
    kind: Cluster
    name: mycluster1
    namespace: mycluster1
    uid: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
  decisions:
  - apiVersion: clusterregistry.k8s.io/v1alpha1
    kind: Cluster
    name: mycluster1
    namespace: mycluster1
    uid: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
  lastUpdateTime: 2020-12-01T13:53:082
  observedGeneration: 1

Example hybrid placement rule YAML

The following YAML content defines example hybrid placement rules:

Example: Pure-Kubernetes resource placement

The following YAML definition is a simple hybrid placement rule for placing pure-Kubernetes based hybrid deployables on a cluster. This hybrid placement rule resembles the non-hybrid pure Kubernetes only placement rule.

apiVersion: core.hybridapp.io/v1alpha1
kind: PlacementRule
metadata:
  name:
spec:
  replicas: 1
  targetLabels:
    matchLabels:
      env: dev

Example: Resource placement with multiple advisors

The following placement rule shows a configuration that includes multiple advisors. The placement of hybrid deployables with this rule is determined through reconciliation of the advisor recommendations. In the following example, the advisor rhacm provides recommendations that are sorted by CPU availability, while the advisor cost provides direct recommendations.

apiVersion: core.hybridapp.io/v1alpha1
kind: PlacementRule
metadata:
  name:
spec:
  deployerType:
  replicas: 1
  targets:
  - name:
  targetLabels:
    matchLabels:
  decisionWeight:
  advisors:
  - name: grc
    type: predicate
    rules:
      policies:
      - name:
  - name: rhacm
    type: priority
    weight: 60
    rules:
      resourceHint:
        type: cpu
  - name: cost
    type: priority
    weight: 50

Example: Virtual machine resource placement

The following hybrid placement rule shows the configuration for placing infrastructure related hybrid deployables, such as for placing and managing virtual machine resources.

apiVersion: core.hybridapp.io/v1alpha1
kind: PlacementRule
metadata:
  name:
spec:
  deployerType: infrastructure
  replicas: 1
  targetLabels:
    matchLabels:
      env: dev
  advisors:
  - name: grc
    type: predicate
    rules:
      policies:
      - name:
  - name: cost
    type: priority
    weight: 50