Cluster

A cluster definition is needed to declare the properties of the IBM Storage Scale container native cluster. The following steps describe creating a Cluster custom resource.

Download cluster CR sample

Download one of our Cluster custom resource (CR) samples hosted on the GitHub repository. Samples are provided as a starting point with some defaults already set in the configuration based on the target environment.

Use one of the following commands to download a copy of the sample.

Red Hat OpenShift Cluster CR

For a Red Hat OpenShift cluster:

curl -fs https://raw.githubusercontent.com/IBM/ibm-spectrum-scale-container-native/v5.2.3.x/generated/scale/cr/cluster/cluster.yaml > cluster.yaml || echo "Failed to download Cluster sample CR"

Kubernetes Cluster CR

For a Kubernetes cluster:

curl -fs https://raw.githubusercontent.com/IBM/ibm-spectrum-scale-container-native/v5.2.3.x/generated/scale/cr/cluster/cluster-k8s.yaml > cluster.yaml || echo "Failed to download Cluster sample CR"

Google Kubernetes Engine Cluster CR

IBM Storage Scale container native deployed in a Google Kubernetes Engine (GKE) is available as a technology preview. For more information, see Support for Google Kubernetes Engine (GKE).

For a Google Kubernetes Engine cluster:

curl -fs https://raw.githubusercontent.com/IBM/ibm-spectrum-scale-container-native/v5.2.3.x/generated/scale/cr/cluster/cluster-gke.yaml > cluster.yaml || echo "Failed to download Cluster sample CR"

NVIDIA InfiniBand RDMA or RDMA over Converged Ethernet (RoCE) cluster CR

For a cluster where NVIDIA InfiniBand or RoCE networking is to be used, start with the cluster.nvidia.yaml file. This sample contains RDMA options that must be configure before applying the resource.

curl -fs https://raw.githubusercontent.com/IBM/ibm-spectrum-scale-container-native/v5.2.3.x/generated/scale/cr/cluster/cluster.nvidia.yaml > cluster.yaml || echo "Failed to download Cluster sample CR"

Apply the Cluster CR

1. Make other changes specific to your installation. For more information on the Cluster specification, see Cluster spec.

2. Set the node labels to identify the nodes in which the IBM Storage Scale container native operator deploys a core pod to. The sample cluster CR defines the selector as scale.spectrum.ibm.com/daemon-selector: "". If you want IBM Storage Scale core pods to run on the Red Hat OpenShift worker nodes, then apply the label to the Kubernetes worker group by using the following command:

   kubectl label nodes -l node-role.kubernetes.io/worker= scale.spectrum.ibm.com/daemon-selector=
   

   For Kubernetes clusters, the command would need a different group label or individual node names depending on the cluster labels for worker nodes. The following is an example of individual nodes when a common worker label is not set.

   kubectl label nodes <node1> <node2> <node3>... scale.spectrum.ibm.com/daemon-selector=
   

3. Apply the cluster yaml by entering the following command:

   kubectl apply -f cluster.yaml
   

   If you apply the Cluster custom resource and do not see any IBM Storage Scale core pods, check the Events in the output of kubectl describe cluster.scale and kubectl describe daemon -n ibm-spectrum-scale for information about what might be causing the problem.

After the cluster resource is defined to Kubernetes, use kubectl edit cluster ibm-spectrum-scale modify properties of the resource.

Cluster spec

The following table describes the properties for the Cluster custom resource spec.

License

The license section allows accepting and choosing the IBM Storage Scale edition to be deployed in the IBM Storage Scale container native cluster. Complete the following activities:

• Review the appropriate license documentation through the URL in the CR.
• Accept the license by specifying true in the license.accept field.
• Supply the edition used in the license.license field.

The sample CR defaults to data-access under the license.license field, indicating IBM Storage Scale Data Access Edition. If you need the IBM Storage Scale Data Management Edition, then change the value in license.license to data-management.

The data-access edition is sufficient if remote mounted file systems are used. To use local file systems, the data-management license is required.

Specifying an edition without proper entitlement results in image pull failures during deployment.

apiVersion: scale.spectrum.ibm.com/v1beta1
kind: Cluster
spec:
  ...
  license:
    accept: true
    license: data-access

Enter the kubectl explain cluster.spec.license command to view more details.

Daemon

The daemon section in the cluster specification specifies configuration for the IBM Storage Scale core pods.

Cluster name override

If you have multiple container native clusters that are attempting to remote mount from a single IBM Storage Scale storage cluster, each client GPFS cluster must have a unique GPFS Cluster name. Depending on the configuration of your Red Hat OpenShift cluster, the default cluster might not be unique and cause failures to mount the file system.

The cluster name is created by taking ibm-spectrum-scale and adding it to the base domain of the Red Hat OpenShift cluster (kubectl get dns cluster -ojson | jq -r '.spec.baseDomain'). If this cluster name is not unique, use the daemon.clusterNameOverride field to override the generated name.

The clusterNameOverride field must be configured before cluster creation. This field cannot be changed after a cluster is created. Setting this field after a cluster is created requires a full cleanup and redeployment of the cluster.

If the following value in the clusterNameOverride field is used:

apiVersion: scale.spectrum.ibm.com/v1beta1
kind: Cluster
...
spec:
  daemon:
    clusterNameOverride: ocp.c2.example.com

As a result, the container native cluster that is created has the following GPFS cluster name:

kubectl exec -n ibm-spectrum-scale \
$(kubectl get pods -lapp.kubernetes.io/name=core -n ibm-spectrum-scale -ojsonpath="{.items[0].metadata.name}") \
-- mmlscluster

The output appears as shown:

mmlssh-4.4# mmlscluster

GPFS cluster information
========================
  GPFS cluster name:         ocp.c2.example.com
...
...

Tolerations

The spec.daemon.tolerations section allows the user to configure extra tolerations to determine where IBM Storage Scale core pods can be scheduled. For more information on tolerations, see Taints and Tolerations in Kubernetes documentation.

The operator rolls out the configured Tolerations to the IBM Storage Scale core pods.

Enter the kubectl explain cluster.spec.daemon.tolerations command to view more details.

Node selectors

Use the daemon.nodeSelector section to configure a nodeSelector to determine where IBM Storage Scale pods can be deployed.

The Operator checks that a node has all defined labels present to deem a node eligible to deploy IBM Storage Scale pods. In the Cluster custom resource sample, the operator deploys IBM Storage Scale pods on nodes with the following label: scale.spectrum.ibm.com/daemon-selector:

apiVersion: scale.spectrum.ibm.com/v1beta1
kind: Cluster
spec:
  ...
  daemon:
    nodeSelector:
      scale.spectrum.ibm.com/daemon-selector: ""

Enter the kubectl explain cluster.spec.daemon.nodeSelector command to view more details.

If no node selector is specified, IBM Storage Scale pods are deployed on all cluster nodes.

Node selector expressions

The daemon.nodeSelectorExpressions can be used as alternative or in addition to daemon.nodeSelector. Enter the kubectl explain cluster.spec.daemon.nodeSelectorExpressions command to view more details.

The Google Kubernetes Engine (GKE) cluster can be spread across multiple node pools. A subset of node pools can be selected with the nodeSelectorExpressions parameter in the Cluster custom resource. The following example selects all nodes of node pools first-pool and second-pool for running Scale pods:

apiVersion: scale.spectrum.ibm.com/v1beta1
kind: Cluster
...
spec:
  daemon:
    nodeSelectorExpressions:
    - key: cloud.google.com/gke-nodepool
      operator: In
      values:
      - first-pool
      - second-pool

Host aliases

It is recommended that proper DNS is configured in your environment.

The daemon.hostAliases section allows for user-defined entries to be added into the IBM Storage Scale CoreDNS service. The IBM Storage Scale CoreDNS service provides the name resolution for the IBM Storage Scale core pods.

If the core pods are unable to resolve the hostname of the servers in the storage cluster by DNS, their hostname and their IP addresses can be specified in the hostAliases as follows:

apiVersion: scale.spectrum.ibm.com/v1beta1
kind: Cluster
spec:
  ...
  daemon:
    hostAliases:
      - hostname: node1.example.com
        ip: 10.0.0.1
      - hostname: node2.example.com
        ip: 10.0.0.2

The IBM Storage Scale CoreDNS service handles name resolution for the core pods. For RemoteCluster CR, the hostname that is provided in the remotecluster.spec.gui.host field must be DNS resolvable and using host aliases is not a valid workaround.

Enter the kubectl explain cluster.spec.daemon.hostAliases command to view more details.

Cluster profile

The daemon.clusterProfile allows the user to set default IBM Storage Scale configuration parameters for the cluster at cluster creation time.

apiVersion: scale.spectrum.ibm.com/v1beta1
kind: Cluster
spec:
  daemon:
    ...
    clusterProfile:
      controlSetxattrImmutableSELinux: "yes"
      enforceFilesetQuotaOnRoot: "yes"
      ignorePrefetchLUNCount: "yes"
      initPrefetchBuffers: "128"
      maxblocksize: 16M
      prefetchPct: "25"
      prefetchTimeout: "30"

Changing the values in the clusterProfile is not supported and must be avoided unless advised by IBM Support. For the two exceptions where changing values in the clusterProfile is supported, see Ephemeral port range and Ephemeral port ranges.

Enter the kubectl explain cluster.spec.daemon.clusterProfile command to view more details.

Ephemeral port range

If the storage cluster configures ephemeral port ranges, you need to set tscCmdPortRange on the container native cluster to match the range.

For example, if the storage cluster is configured to use port range of 60000-61000, set this value under the clusterProfile section in the Cluster custom resource.

apiVersion: scale.spectrum.ibm.com/v1beta1
kind: Cluster
spec:
  ...
  daemon:
    clusterProfile:
      ...
      tscCmdPortRange: "60000-61000"

Roles

The daemon.roles section in the Cluster spec allows the user to fine-tune memory and CPU requests by using the resources object on the nodes that are part of specific IBM Storage Scale roles. For more information, see Resource Management for Pods and Containers in Kubernetes documentation.

client role

The default node role is client when the node is not labeled with an AFM role.

For the client role, the configuration recommendation for requests is 2 CPU and 4 GiB. On systems with many CPU cores, large memory, and/or high-speed network, the storage performance might increase with higher values. Therefore, encryption and compression of PersistentVolumes result in higher CPU load higher resource values can be beneficial. On smaller systems and/or applications with low I/O workloads, 1 CPU and 2 GiB can be set.

For the AFM role, the resource values are considerabily higher and should be considered for proper performance. For more information, see General recommendations for AFM gateway node configuration

Low resource configurations might yield poor performance.

The resource properties of the IBM Storage Scale core pods are as follows:

• By default, role resources are set to 25% of the capacity of the nodes. The daemon.roles.resources parameter in cluster spec is identical to the requests parameter in the Kubernetes pod spec.
• By default, role limits are set to 50% of the capacity of the nodes.

All node resources must be homogeneous for the same role.

To set memory and CPU requests and limits for the client role, specify the request values under daemon.roles.resources and the limit values under daemon.roles.limits with kubectl edit cluster command.

Limits must have at least the double value as the requests when specified. See the example (kubectl describe cluster):

Changing these values on an existing cluster causes a rolling drain of nodes and pod restarts.

apiVersion: scale.spectrum.ibm.com/v1beta1
kind: Cluster
spec:
  ...
  daemon:
    roles:
    - name: client
      resources:
        cpu: "2"
        memory: 4Gi
      limits:
        cpu: "4"
        memory: 8Gi

For s390x, the sample cluster custom resource ships with "4Gi" memory request. You must reduce the memory request to "2Gi" if the hardware does not have enough physical memory.

Enter the kubectl explain cluster.spec.daemon.roles command to view more details.

Update

The daemon.update section specifies the configuration of how the IBM Storage Scale core pods are updated. By default the core pods are updated in a rolling node-by-node fashion. Therefore, in a cluster with many nodes the update takes a long time. To speed up, multiple core pods can be updated in parallel.

The daemon.update.maxUnavailable field allows update of multiple core pods in parallel. For example, maxUnavailable: 3 causes that up to 3 nodes are cordoned and drained by the operator, and 3 core pods are updated in parallel.

apiVersion: scale.spectrum.ibm.com/v1beta1
kind: Cluster
spec:
  ...
  daemon:
    update:
      maxUnavailable: 3

Instead of a total number a percentage can be specified. For example, maxUnavailable: 33% causes that up to a third of all core pods are updated at the same time. In this case, all core pods are updated in at least 3 update cycles.

apiVersion: scale.spectrum.ibm.com/v1beta1
kind: Cluster
spec:
  ...
  daemon:
    update:
      maxUnavailable: 33%

The operator keeps the container native cluster operational during the update. At least 3 update cycles are needed because most quorum core pods must always be running to prevent downtime of the container native cluster. For example, with 7 quorum pods, first up to 3 quorum pods are updated. Then, the next 3 quorum pods are updated. Finally, the last quorum pod is updated. Therefore, it's typically not useful to specify maxUnavailable: 100% because the pods are still updated in three cycles: The first 3 quorum pods plus all nonquorum pods, then the next 3 quorum pods and the last quorum pod. Therefore, maxUnavailable: 33% is recommended as an optimal configuration when the update speed must be short.

In addition to quorum, other characteristics limit the number of core pods to be updated in parallel. As a result, it can happen that fewer pods than defined in maxUnavailable are updated in parallel.

The update can be stopped by specifying paused: true.

apiVersion: scale.spectrum.ibm.com/v1beta1
kind: Cluster
spec:
  ...
  daemon:
    update:
      maxUnavailable: 3
      paused: true

The daemon.update.pools section is used to define update pools. Use update pools to specify the update configuration for a subset of core pods. The core pods for the pool are selected with a node selector.

apiVersion: scale.spectrum.ibm.com/v1beta1
kind: Cluster
spec:
  ...
  daemon:
    update:
      maxUnavailable: 2
      pools:
      - name: fast
        nodeSelector:
          matchLabels:
            fast-update: ""
        maxUnavailable: 10

In the example, all core pods on nodes with label fast-update: "" are part of the fast update pool. Up to 10 core pods in this pool can be updated in parallel. For other pods (without node label fast-update: "") the maxUnavailable: 2 matters, which allows to update up to 2 pods in parallel. Each core pod must belong to maximum one update pool.

Using the matchExpressions is another way for selecting nodes for a pool.

apiVersion: scale.spectrum.ibm.com/v1beta1
kind: Cluster
spec:
  ...
  daemon:
    update:
      maxUnavailable: 1
      pools:
      - name: expressionSample
        nodeSelector:
          matchExpressions:
          - key: scale.spectrum.ibm.com/designation
            operator: DoesNotExist
          - key: topology.kubernetes.io/zone
            operator: In
            values:
            - us-east1-a
            - us-east1-b
        maxUnavailable: 4

Only nodes that run an IBM Storage Scale pod are considered for update pools. Enter the kubectl explain cluster.spec.daemon.update.pools.nodeSelector command to view more details.

Check daemon status to verify whether the expected nodes are selected with the node selector. The nodes that do not belong to any update pool are listed with pool name default.

# kubectl describe daemon ibm-spectrum-scale -n ibm-spectrum-scale
...
Status:
  ...
  Update:
    Pools:
      Name:        default
      Node Count:  3
      Nodes:       worker0.example.com, worker1.example.com, worker2.example.com
      Name:        example-pool
      Node Count:  5
      Nodes:       worker3.example.com, worker4.example.com, worker5.example.com, worker6.example.com, worker7.example.com

Update pools with prefix mpc/ in their name reference to an Red Hat OpenShift MachineConfigPool. In this case, the parameters maxUnavailable, nodeSelector, and paused are copied from the Red Hat OpenShift MachineConfigPool and the update pool configuration prevents these parameters.

apiVersion: scale.spectrum.ibm.com/v1beta1
kind: Cluster
spec:
  ...
  daemon:
    update:
      pools:
      - name: "mcp/worker"

apiVersion: machineconfiguration.openshift.io/v1
kind: MachineConfigPool
metadata:
  name: worker
spec:
  ...
  maxUnavailable: 1
  nodeSelector:
    matchLabels:
      node-role.kubernetes.io/worker: ""
  paused: false

Parallel Update recommendations

The maxUnavailable parameter allows to enable parallel update. For more information, see Update. When parallel update is enabled, it makes sense to reduce the maxUnavailable number for a subset of the nodes to increase the availability.

Quorum

By default, the operator updates as many quorum pods in parallel as possible if parallel update is enabled. With 5 quorum pods, then 2 quorum pods are updated in parallel and the remaining 3 quorum pods keep running. This causes a container native cluster downtime if one more quorum pod is going down unexpectedly during an update. If availability of the container native cluster is more important than update speed, create the following update pool for quorum pods that allows to update one quorum pod at any time.

apiVersion: scale.spectrum.ibm.com/v1beta1
kind: Cluster
spec:
  ...
  daemon:
    update:
      maxUnavailable: 5
      pools:
      - name: quorum
        nodeSelector:
          matchLabels:
            scale.spectrum.ibm.com/designation: quorum
        maxUnavailable: 1

If IBM Storage Scale container native is deployed in Google Kubernetes Engine (GKE), it's important to create the quorum update pool with maxUnavailable: 1 as described in previous sample.

If IBM Storage Scale container native is deployed in Google Kubernetes Engine (GKE), the surge upgrade strategy is used for GKE node pool upgrades. For more information, see Surge upgrades. For each node pool, in GKE a corresponding update pool must be created. The sum of the maxUnavailable and the maxSurge parameters for the surge configuration of a GKE node pool should match to the maxUnavailable parameter of the update pool.

For example, in the following GKE surge upgrade configuration

• GKE node pool first-pool has maxUnavailable=3 and maxSurge=2
• GKE node pool second-pool has maxUnavailable=2

the corresponding update pool configuration looks like

apiVersion: scale.spectrum.ibm.com/v1beta1
kind: Cluster
spec:
  ...
  daemon:
    update:
      pools:
      - name: first-pool
        nodeSelector:
          matchLabels:
            cloud.google.com/gke-nodepool: first-pool
        maxUnavailable: 5
      - name: second-pool
        nodeSelector:
          matchLabels:
            cloud.google.com/gke-nodepool: second-pool
        maxUnavailable: 2

Local file system

Local file systems use disks that have a shared connection to many nodes. Even though the operator avoids that all connection nodes of a disk are updated in parallel, it makes sense to reduce the number of disk connection nodes. If availability of the local file systems and the I/O performance is more important than update speed, create an update pool for connection nodes. The following update pool causes that connection nodes are updated one at a time, if the shared disk connection nodes are labeled with connection=true.

apiVersion: scale.spectrum.ibm.com/v1beta1
kind: Cluster
spec:
  ...
  daemon:
    update:
      maxUnavailable: 5
      pools:
      - name: diskConnection
        nodeSelector:
          matchLabels:
            connection: true
        maxUnavailable: 1

For more information, see Local file system.

Grafana bridge

The grafanaBridge section allows the user to enable the deployment of the IBM Storage Scale bridge for Grafana application. This tool allows the IBM Storage Scale container native users to perform performance monitoring for IBM Storage Scale devices, by using third-party applications such as Grafana or Prometheus software. For more information, see IBM Storage Scale bridge for Grafana.

Specify grafanaBridge: {} to enable Grafana Bridge:

apiVersion: scale.spectrum.ibm.com/v1beta1
kind: Cluster
spec:
  ...
  grafanaBridge: {}

Grafana bridge is optional

Enter the kubectl explain cluster.spec.grafanabridge command to view more details.

Prometheus

You can expose the IBM Storage Scale performance metrics to a Prometheus instance that can be configured by Red Hat OpenShift to monitor the project. Using this allows monitoring of both Red Hat OpenShift components and IBM Storage Scale container native resources in a single place. For an end-to-end guide on this configuration, see Setup Red Hat OpenShift Monitoring Stack with IBM Storage Scale container native.

Set enablePrometheusExporter property to true

apiVersion: scale.spectrum.ibm.com/v1beta1
kind: Cluster
spec:
  ...
  grafanaBridge:
    enablePrometheusExporter: true

Pmcollector Volumes for Performance Data

The pmcollector requires persistent volumes for storing performance data. Depending on setup, there are two modes for configuring these volumes:

1. Local Persistent Volumes: When no storage class is defined in pmcollector section, the default storage class is used and local volumes are created.

2. Specific Storage Class for Persistent Volumes: If a storage class is specified in pmcollector section, Persistent Volume Claims (PVCs) are created using that storage class. This is commonly used in cloud environments, where storage management is handled by the cloud provider.

    apiVersion: scale.spectrum.ibm.com/v1beta1
    kind: Cluster
    spec:
      ...
      pmcollector:
        storageClass: "your-storage-class"
   

Enter the kubectl explain cluster.spec.pmcollector command to view more details.

Modifying storageClass field deletes existing PVCs and create new ones with the updated storage class. This leads to the loss of the historical performance data.

Use a storage class (second option above) when IBM Storage Scale container native is deployed in Google Kubernetes Engine (GKE).

CSI Configuration

The csi section allows the user to define the configuration of the Container Storage Interface (CSI).

By default, the CSI pods are scheduled on worker nodes. The csi.sidecar.nodeSelector can be used to schedule the IBM Storage Scale Container Storage Interface (CSI) sidecar pods (external-attacher, external-provisioner, external-snapshotter, and external-resizer) onto specific set of nodes.

The CSI sidecar nodeSelector section allows specifying a Kubernetes Node Selector. The nodeSelector must select at least two nodes to provide high availability of CSI sidecar pods.

apiVersion: scale.spectrum.ibm.com/v1beta1
kind: Cluster
spec:
  ...
  csi:
    sidecar:
      nodeSelector:
        key: value

Enter the kubectl explain cluster.spec.csi command to view more details.

CSI configuration is optional

Infrastructure nodes

The GUI pods, pmcollector pods, and Grafana bridge pods can be placed onto Red Hat OpenShift infrastructure nodes. At least two infrastructure nodes are needed because two replicas of GUI and pmcollector pods must run on different infrastructure nodes.

GUI, pmcollector, and Grafana bridge pods

The gui, pmcollector and grafanaBridge sections allow to specify a Kubernetes Node Selector and Kubernetes Taints and Tolerations.

If the Red Hat OpenShift infrastructure nodes are labeled with node-role.kubernetes.io/infra="" and have the node-role.kubernetes.io/infra:NoSchedule and node-role.kubernetes.io/infra:NoExecute taints, add the following lines to run the gui, pmcollector and grafanaBridge sections:

apiVersion: scale.spectrum.ibm.com/v1beta1
kind: Cluster
spec:
  ...
  gui:
    nodeSelector:
      node-role.kubernetes.io/infra: ""
    tolerations:
    - effect: NoSchedule
      key: node-role.kubernetes.io/infra
      operator: Exists
    - effect: NoExecute
      key: node-role.kubernetes.io/infra
      operator: Exists
  pmcollector:
    nodeSelector:
      node-role.kubernetes.io/infra: ""
    tolerations:
    - effect: NoSchedule
      key: node-role.kubernetes.io/infra
      operator: Exists
    - effect: NoExecute
      key: node-role.kubernetes.io/infra
      operator: Exists
  grafanaBridge:
    nodeSelector:
      node-role.kubernetes.io/infra: ""
    tolerations:
    - effect: NoSchedule
      key: node-role.kubernetes.io/infra
      operator: Exists
    - effect: NoExecute
      key: node-role.kubernetes.io/infra
      operator: Exists

CSI sidecar pods

The csi section allow to specify a Kubernetes Node Selector.

If the Red Hat OpenShift infrastructure nodes are labeled with node-role.kubernetes.io/infra="", add the following lines to run CSI sidecar pods on infrastructure nodes:

apiVersion: scale.spectrum.ibm.com/v1beta1
kind: Cluster
spec:
  ...
  csi:
    sidecar:
      nodeSelector:
        node-role.kubernetes.io/infra: ""

The CSI sidecar pods tolerate the NoExecute and NoSchedule taint. Therefore, the sidecar pods can run on infrastructure nodes even if these taints are present.

If parallel updates (maxUnavailable > 1) are enabled along with placing CSI sidecar pods on infrastructure nodes, create an update pool for nodes with maxUnavailable = 1. This ensures that only one core pod is updated at a time, therefore CSI is operational on all other nodes where the core pods are running.

apiVersion: scale.spectrum.ibm.com/v1beta1
kind: Cluster
spec:
  ...
  daemon:
    update:
      pools:
      - name: infra
        nodeSelector:
          matchLabels:
            node-role.kubernetes.io/infra: ""
        maxUnavailable: 1

IBM Storage Scale core pods

To use infrastructure nodes, it is required that IBM Storage Scale core pods also run on the infrastructure nodes.

• Label all worker nodes and infrastructure nodes with a common label. The sample Cluster custom resource defines the node selector: scale.spectrum.ibm.com/daemon-selector: "". To use this selector, apply the following labels to worker and infrastructure nodes:

    kubectl label nodes --selector node-role.kubernetes.io/worker scale.spectrum.ibm.com/daemon-selector=""
  

    kubectl label nodes --selector node-role.kubernetes.io/infra scale.spectrum.ibm.com/daemon-selector=""
  

The IBM Storage Scale core pods tolerate the NoExecute and NoSchedule taint. Therefore, the core pods run on infrastructure nodes even if these taints are present.

Network policies

The cluster.spec.networkPolicy object indicates whether the container native operator creates and manages the network policy rules within the IBM Storage Scale container native namespaces. The default sample cluster CR enables this option: networkPolicy: {}.

Modifying the network policies that are owned by the Cluster controller gets overwritten by the operator.

Enabling and disabling network policy management

With IBM Storage Scale container native v5.1.9 or later, the sample cluster custom resource enables network policies by default.

If network policies are not created by the operator, use the following command to patch the Cluster spec:

kubectl patch cluster.scale.spectrum.ibm.com ibm-spectrum-scale --type='json' \
-p='[{"op": "add", "path": "/spec/networkPolicy", "value":{}}]'

If you find that the network policy rules are interfering with functions of the container native cluster, disable the option in the Cluster spec with the following command:

kubectl patch cluster.scale.spectrum.ibm.com ibm-spectrum-scale --type='json' \
-p='[{"op": "remove", "path": "/spec/networkPolicy"}]'

Delete the network policies with the following commands:

kubectl delete networkpolicy -n ibm-spectrum-scale-operator --all
kubectl delete networkpolicy -n ibm-spectrum-scale --all
kubectl delete networkpolicy -n ibm-spectrum-scale-dns --all

Allowing communication between trusted namespaces

To allow ingress and egress communication to and from a trusted namespace within your Red Hat OpenShift cluster, label the namespace with the label: scale.spectrum.ibm.com/networkpolicy=allow.

To enable communication, add the label to the namespace:

kubectl label namespace <TARGET_NAMESPACE> scale.spectrum.ibm.com/networkpolicy=allow

To disable communication, remove the label on the namespace:

kubectl label namespace <TARGET_NAMESPACE> scale.spectrum.ibm.com/networkpolicy-

Cluster status

Status Conditions can be viewed as a snapshot of the current and most up-to-date status of a Cluster.

• The Success condition is set to True if the Cluster is successfully configured.