Modifying deployment to be Zone Aware

About this task

Currently, the file-uploader deployment is not zone aware and can schedule all the pods in the same zone. In this case, when there is a site outage then the application is no longer available. For more information, see Controlling pod placement by using pod topology spread constraints.

Procedure

  1. Add the pod placement rule in the application deployment configuration to make the application zone aware.
    1. Run the following command, and review the output:
      oc get deployment file-uploader -o yaml -n my-shared-storage | less
      Example Output:
      [...]
      spec:
        progressDeadlineSeconds: 600
        replicas: 4
        revisionHistoryLimit: 10
        selector:
          matchLabels:
            deployment: file-uploader
        strategy:
          rollingUpdate:
            maxSurge: 25%
            maxUnavailable: 25%
          type: RollingUpdate
        template:
          metadata:
            annotations:
              openshift.io/generated-by: OpenShiftNewApp
            creationTimestamp: null
            labels:
              deployment: file-uploader
            spec:  # <-- Start inserted lines after herecontainers:  # <-- End inserted lines before here
              - image: image-registry.openshift-image-registry.svc:5000/my-shared-storage/file-uploader@sha256:a458ea62f990e431ad7d5f84c89e2fa27bdebdd5e29c5418c70c56eb81f0a26b
                imagePullPolicy: IfNotPresent
                name: file-uploader
      [...]
    2. Edit the deployment to use the topology zone labels.
      oc edit deployment file-uploader -n my-shared-storage
      Add add the following new lines between the Start and End (shown in the output in the previous step):
      [...]
          spec:
            topologySpreadConstraints:
              - labelSelector:
                  matchLabels:
                    deployment: file-uploader
                maxSkew: 1
                topologyKey: topology.kubernetes.io/zone
                whenUnsatisfiable: DoNotSchedule
              - labelSelector:
                  matchLabels:
                    deployment: file-uploader
                maxSkew: 1
                topologyKey: kubernetes.io/hostname
                whenUnsatisfiable: ScheduleAnyway
            nodeSelector:
              node-role.kubernetes.io/worker: ""
            containers:
      [...]
      Example output:
      deployment.apps/file-uploader edited
  2. Scale down the deployment to zero pods and then back to four pods. This is needed because the deployment changed in terms of pod placement.
    Scaling down to zero pods
    oc scale deployment file-uploader --replicas=0 -n my-shared-storage
    Example output:
    deployment.apps/file-uploader scaled
    Scaling up to four pods
    oc scale deployment file-uploader --replicas=4 -n my-shared-storage
    Example output:
    deployment.apps/file-uploader scaled
  3. Verify that the four pods are spread across the four nodes in datacenter1 and datacenter2 zones.
    oc get pods -o wide -n my-shared-storage | egrep '^file-uploader'| grep -v build | awk '{print $7}' | sort | uniq -c
    Example output:
    1 perf1-mz8bt-worker-d2hdm
       1 perf1-mz8bt-worker-k68rv
       1 perf1-mz8bt-worker-ntkp8
       1 perf1-mz8bt-worker-qpwsr
    Search for the zone labels used.
    oc get nodes -L topology.kubernetes.io/zone | grep datacenter | grep -v master
    Example output:
    perf1-mz8bt-worker-d2hdm   Ready    worker   35d   v1.20.0+5fbfd19   datacenter1
    perf1-mz8bt-worker-k68rv   Ready    worker   35d   v1.20.0+5fbfd19   datacenter1
    perf1-mz8bt-worker-ntkp8   Ready    worker   35d   v1.20.0+5fbfd19   datacenter2
    perf1-mz8bt-worker-qpwsr   Ready    worker   35d   v1.20.0+5fbfd19   datacenter2
  4. Use the file-uploader web application by using your browser to upload new files.
    1. Find the route that is created.
      oc get route file-uploader -n my-shared-storage -o jsonpath --template="http://{.spec.host}{'\n'}"
      Example Output:
      http://file-uploader-my-shared-storage.apps.cluster-ocs4-abdf.ocs4-abdf.sandbox744.opentlc.com
    2. Point your browser to the web application by using the route in the previous step.

      The web application lists all the uploaded files and offers the ability to upload new ones and download the existing data.

    3. Select an arbitrary file from your local computer and upload it to the application.
      1. Click Choose file to select an arbitrary file.
      2. Click Upload.
    4. Click List uploaded files to see the list of all currently uploaded files.