Managing PX compute pod placement using Red Hat OpenShift labels

Use PX compute pod placement configuration with Red Hat® OpenShift® to control where pods run within your cluster to meet operational, security, and compliance requirements. By default, DataStage PX compute pods might not be evenly distributed across availability zones, which can affect high availability and infrastructure requirements. You can spread pods across multiple availability zones to ensure high availability and service continuity if one zone fails. For security and compliance purposes, you can run pods only on nodes with specific firewall rules, or restrict DataStage workloads to nodes with particular network access. You can also distribute pods across nodes with specific hardware configurations to optimize performance and meet regulatory requirements for workload placement.

Compute pod placement configuration

  1. Use the following label nodes with availability zone information:
    oc label node <node-ip-1> topology.kubernetes.io/zone=us-south-az1
oc label node <node-ip-2> topology.kubernetes.io/zone=us-south-az1
oc label node <node-ip-3> topology.kubernetes.io/zone=us-south-az2
oc label node <node-ip-4> topology.kubernetes.io/zone=us-south-az2
oc label node <node-ip-5> topology.kubernetes.io/zone=us-south-az3
oc label node <node-ip-6> topology.kubernetes.io/zone=us-south-az3

    If you have multiple PXRuntime instances, use custom names for your labels, for example: my-zone-label=az1.

  2. Confirm that the labels are applied.
    oc get nodes -L topology.kubernetes.io/zone
  3. Configure PXRemoteEngine CR for custom labels:
    spec:
  azTopologyKey: my-zone-label
    Note: For the default label topology.kubernetes.io/zone, no configuration is needed. The azTopologyKey parameter defaults to topology.kubernetes.io/zone.
  4. Confirm that the compute pods are distributed across zones.
    oc get pods -n <namespace> -o wide -l app.kubernetes.io/component=px-compute

Examples

The PXRuntime CR spreads pods evenly among zones indicated by the topology label:

  • 6 nodes across 3 AZs: when you deploy three compute pods, each one is placed on a node with the matching AZ label in its zone. For example, the pods might run on node-ip-1, node-ip-3, and node-ip-5.
  • 6 compute pods: when you deploy six compute pods, each pod is placed on a separate node, with the pods distributed across all zones.