Resizing a PVC for Postgres on OpenShift by moving content

Resize a Persistent Volume Claim (PVC) for Postgres on OpenShift by creating a new, larger PVC, and then moving content between the PVCs.

Before you begin

This task only applies to the PVC used by Postgres.

About this task

Use this procedure when the storage class does not support volume expansion.

Procedure

Create a new temporary PVC. Ensure the requested storage size is more than the original PVC. For example:

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: temp-pvc
spec:
  storageClassName: rook-ceph-block
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 10Gi

Determine the primary Postgres pod.

oc get pods --selector role=master | grep -v bootstrapselector=role=master

Gracefully shutdown Postgres pods.
1. Exec into the Postgres primary pod.
```
oc exec -it <postgres-pod> -- bash
```
2. Pause the Patroni process. Note: you can skip this step if using the one replica profile.
```
patronictl pause
```
3. Stop the Postgres process gracefully.
```
pg_ctl stop -D /pgdata/<cluster-name>
```
  For example:
  - cluster-name is usually the name of the cluster. If primary pod name is m1-1b12bf81-postgres-58dfcc5f97-fsvvt, cluster-name is m1-1b12bf81-postgres.
  - There will be a folder named cluster-name located under /pgdata/.
  - In this example, the command is:
```
pg_ctl stop -D /pgdata/m1-1b12bf81-postgres
```

Determine the PVC linked to the primary Postgres pod, and obtain the claimName of the PVC.

oc get pod <postgres-pod> -o json | jq '.spec.volumes[] | select ( has ("persistentVolumeClaim"))'

Scale down the Postgres deployment.

oc get deploy | grep postgres | grep -v bouncer | grep -v bootstrap | grep -v backrest | grep -v operator

If you are using n3 profile then you find 3 deployments. Scale them down:

oc scale deploy <deployment-name> --replicas=0

Start a BusyBox pod with the source and destination PVC volumes mounted, to swap the content between PVCs. For example:

Note: After you run this command, you might see a warning like the one shown after this. You can ignore this. Warning: would violate PodSecurity "restricted:v1.24": allowPrivilegeEscalation != false (container "debug" must set securityContext.allowPrivilegeEscalation=false), unrestricted capabilities (container "debug" must set securityContext.capabilities.drop=["ALL"]), runAsNonRoot != true (pod or container "debug" must set securityContext.runAsNonRoot=true), seccompProfile (pod or container "debug" must set securityContext.seccompProfile.type to "RuntimeDefault" or "Localhost")
```
apiVersion: v1
kind: Pod
metadata:
  name: swap-pvc
spec:
  volumes:
    - name: volume-source
      persistentVolumeClaim:
        claimName: m1-mgmt-35817114-postgres-wal
    - name: volume-destination
      persistentVolumeClaim:
        claimName: temp-pvc
  containers:
    - name: debug
      image: busybox
      command: ['sleep', '3600']
      volumeMounts:
        - name: volume-source
          mountPath: /data-source
        - name: volume-destination
          mountPath: /data-destination 
```
Exec to the BusyBox pod, and run the following commands:
```
oc exec -it <busybox-pod-name> -- sh
```
Copy content to the temporary PVC. For example, using the mountPath values shown in Step 6:
```
cp -r data-source/ data-destination/
```
Exit the BusyBox pod.
```
exit
```
Delete the BusyBox pod.
```
oc delete pod <busybox-pod-name>
```
Delete the original PVC.
```
oc delete pvc <claimName>
```

Create an original PVC with the same name but with increased size. For example:

kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: m1-mgmt-35817114-postgres-wal
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 20Gi
  storageClassName: rook-ceph-block
  volumeMode: Filesystem

To start the BusyBox pod again, run the following command:
```
oc exec -it <busybox-pod> -- sh
```
Copy the content from the destination to the source PVC.
```
cp -r data-destination/ data-source/
```
Exit the BusyBox pod.
```
exit
```
Delete the BusyBox pod.
```
oc delete pod <busybox-pod>
```
Delete the temporary PVC that you created for swapping the content.
```
oc delete pvc temp-pvc
```

Scale up the Postgres deployment.

oc get deploy | grep postgres | grep -v bouncer | grep -v bootstrap | grep -v backrest | grep -v operator

If you are using the n3 profile then you find 3 deployments. Scale them up:

oc scale deploy <deployment-name> --replicas=1

If using the n3 profile, resume the Patroni process. If using the n1 profile, skip this step.
1. Identify the Postgres pods.
```
oc get pods --selector role
```
2. Exec into any Postgres pod.
```
oc exec -it <postgres-pod> -- bash
```
3. Resume Patroni.
```
patronictl resume
```