Special cases for workload controller resize actions

Edit online

Resize actions for certain types of workloads may be blocked or set to recommend-only, which means that they are currently not executable in Turbonomic. These actions usually require a dependent action or an external configuration to be completed first. For example, a workload controller resize up action may be temporarily blocked if there is insufficient resources in the corresponding node or namespace. In this case, a node provision or namespace quota resize action must be completed first.

Blocking of action execution due to insufficient quotas

Turbonomic treats quotas defined in a namespace as constraints when making resize decisions. If existing actions would exceed the namespace quotas, Turbonomic recommends actions to resize up the affected namespace quota.

Note:

For more information about namespace quotas, see Namespace monitored resources.

The action details for a workload controller include descriptions of the affected container spec entities, and how the resources will change for each. If the resize exceeds current namespace quotas, then Turbonomic blocks the workload controller action. The action details list the Namespace actions that block execution of this resize in the Related Actions list.

Recommend-only resize actions for container specs that are unresizable in Turbonomic

For a container spec to be resizable in Turbonomic, it must be controlled by one of the following built-in APIs for managing workloads:

DaemonSet
Deployment
DeploymentConfig
ReplicaSet
ReplicationController
StatefulSet

For container specs that are controlled by other APIs (for example by CronJob or Job), Turbonomic adds the affected container specs in each cluster to a group called Unresizable ContainerSpecs. This group applies a Turbonomic read-only policy called Unresizable ContainerSpecs Resize Recommend Only, which sets the action acceptance mode for container spec resize actions to Recommend. This means that you can only execute resize actions outside of Turbonomic.

Note:

For container specs that are controlled by Custom Resources (or operators), see the next section.

Recommend-only resize actions for operator-controlled workloads

For workloads in a cluster that are controlled by an operator, Kubeturbo automatically creates a group called Operator Controlled ContainerSpecs. This group applies a Turbonomic read-only policy called Operator Controlled ContainerSpecs Resize Recommend Only [cluster_name], which sets the action acceptance mode for resizes to Recommend. This means that you can only execute resizes outside of Turbonomic.

Turbonomic prevents or blocks the creation of a conflicting policy that automates resize actions.

To execute resize actions for operator-controlled workloads, perform any of the following tasks:

Deploy operator resource mapping (ORM) , if needed as not all operators control resource updates in the workloads they manage.

Add operator-controlled workloads to an exclusion list that is defined in a configmap.

For OperatorHub or operator deployments of Kubeturbo, update the kubeturbo-release Custom Resource (CR) by following these steps.

Edit kubeturbo-release.

kubectl edit kubeturbo kubeturbo-release

In the exclusionDetectors section, use one or both of the following parameters to exclude workloads:
- operatorControlledWorkloadsPatterns – Specify workload name patterns.
- operatorControlledNamespacePatterns – Specify namespace name patterns.
```
spec:
  exclusionDetectors:
    operatorControlledWorkloadsPatterns:
    - turbon.*
    - testing-.*
    operatorControlledNamespacePatterns:
    - turbonomic
    - gke-.*
```

Verify that configmap was updated.

There are two ways to verify the updates.

Check the configmap.

kubectl get configmap turbo-config-kubeturbo-release -o yaml

In the output, the turbo-autoreload.config section shows the updates.

apiVersion: v1
data:
  turbo-autoreload.config: |-
    {
     ...
      },
      "exclusionDetectors": {
        "operatorControlledWorkloadsPatterns": ["turbon.*","testing-.*"],
        "operatorControlledNamespacePatterns": ["turbonomic","gke-.*"]
      }
    }

Check the Kubeturbo logs.

kubectl logs kubeturbo-release-7cb4b886c5-glv9b | grep exclusion

Log entries shows the updates.

I1122 14:43:07.603989 dynamic_config.go:119] Operator controlled workload exclusion set to: []
I1122 14:43:07.604013 dynamic_config.go:125] Operator controlled namespace exclusion set to: [turbonomic]
I1122 19:14:52.314970 dynamic_config.go:119] Operator controlled workload exclusion set to: [turbon.* testing-.*]
I1122 19:14:52.315096 dynamic_config.go:125] Operator controlled namespace exclusion set to: [turbonomic gke-.*]

Handling of resize actions for sidecar container specs

A container platform service might include sidecar container specs to provide additional services to a running pod, such as security or logging services.

Injected sidecars are containers that run within a pod but were not explicitly specified in the deployment. These containers are often added by service meshes like Istio using a mutating webhook admission controller. For more information, see the Istio documentation.

Sidecars injected at pod creation cannot be updated from the parent workload controller because limits or requests are controlled by another workload controller. Because the sidecar specification does not exist, a resize action fails.

Resize actions are controlled in the following ways:

To prevent action execution for injected sidecars, Turbonomic adds the sidecars in each a cluster to a group called Injected Sidecars/All ContainerSpecs. This group applies a Turbonomic read-only policy called Injected Sidecars/All ContainerSpecs Resize Recommend Only [cluster_name], which sets the action acceptance mode for resizes to Recommend by default. This means that you can only execute resizes outside of Turbonomic. The parent workload controller will continue to resize non-sidecar container specs as usual.
To prevent action generation for injected sidecars, ensure that both Kubeturbo and Kubeturbo operator are updated to version 8.16.1 (or later) and use the configuration setting, disableActionGeneration.

This configuration setting is backward compatible. If omitted, the system defaults to the recommended mode for action generation in injected sidecars. To disable action generation, set the configuration to true. By default, when this setting is not specified, action generation remains enabled in accordance with the recommended behavior.

For granular control, the setting applies to individual clusters as they are added to the Kubeturbo configuration. Kubeturbo automatically detects the setting and does not require a restart. The setting takes effect in the next discovery cycle.

To update the existing Kubeturbo deployment with the setting, follow these steps.

Scenario 1: Kubeturbo was not deployed through an operator.
1. Update the Kubeturbo image to 8.16.1 or later in the Kubeturbo deployment.
```
kubectl edit deploy kubeturbo-release
```
```
.spec.template.spec.containers[0].image
```
```
image: icr.io/cpopen/turbonomic/kubeturbo:8.16.1
```
2. Update the Kubeturbo configMap by adding a new key policySettings in the turbo-autoreload.config entry.
```
kubectl edit cm turbo-config
```
```
.data.turbo-autoreload.config
```
```
"policySettings": {
           "containers": {
             "sidecars": {"disableActionGeneration": true}
          }
```
  For a sample configMap, see this YAML resource.
Scenario 2: Kubeturbo was deployed through an operator.
1. Update the Kubeturbo Custom Resource Definition (CRD).
```
kubectl apply -f https://raw.githubusercontent.com/IBM/turbonomic-container-platform/main/kubeturbo/operator/kubeturbo_crd.yaml
```
2. In the Kubeturbo operator deployment, update the Kubeturbo operator image to 8.16.1 or later.
```
kubectl edit deploy kubeturbo-operator
```
```
.spec.template.spec.containers[0].image
```
```
.spec.template.spec.containers[0].image
```
3. Update the Kubeturbo CR.
```
kubectl edit kubeturbo kubeturbo-release
```
4. In the CR, specify the Kubeturbo image tag as 8.16.1 or later.
5. In the CR, find spec and then add a section for policySettings.
```
spec:
  policySettings:
    containers:
      sidecars:
        disableActionGeneration: true
```
  For a sample Kubeturbo CR, see this YAML resource.

Handling `LimitRange` constraints

LimitRange objects are namespace scoped objects that define the following resource specifications for every pod or container in the namespace.

Minimum CPU/memory requests and limits
Maximum CPU/memory requests and limits
Default CPU/memory requests and limits
(Optional) Storage limits for PersistentVolumeClaims

Before Kubeturbo executes a workload resize action, it evaluates the proposed resources against the active LimitRange. It injects any defaults specified by the LimitRange if the workload controller has not set explicit values and then checks the new CPU and memory settings against the defined minimum and maximum specifications. If any rule is violated, Kubeturbo marks the action as failed, returns a LimitRange violation error, and then leaves the workload controller unchanged.

Kubeturbo never edits LimitRange objects and does not surface them as entities in the Turbonomic user interface. They are referenced only at execution time. Storage specific constraints in LimitRange objects are currently ignored.

Disabled resize actions for system namespace workloads

Turbonomic disables resize actions for workloads in well-known system namespaces, such as those starting with kube-.*, openshift-.*, or cattle.*. A setting in the Kubeturbo configmap automatically creates a group called System Namespaced ContainerSpecs in each cluster. This group applies a Turbonomic read-only policy called System Namespaced ContainerSpecs Resize Disabled [cluster_name], which sets the action acceptance mode for resizes to Disabled.

For OperatorHub or operator deployments of Kubeturbo, update the kubeturbo-release Custom Resource (CR).