Using an sftp server for backup files on VMware

The sftp backup script enables you to upload and download Postgres backups for the Management Subsystem to and from an sftp server.

The script resides on the Postgres Backrest Repo Pod known as the Backup and Restore Pod. The script is invoked by using the Kubernetes interactive shell (exec) on the script with appropriate parameters. The script can use password or SSH key for sftp server authentication.

Backup files stored on the sftp server are named according to ID of the latest backup available in:

/backrestrepo/<pg cluster>-backrest-shared-repo/backup/db/.

The names of these backups align with IDs found in kubectl get mgmtb lists. For example:

$ kubectl get mgmtb
NAME                STATUS   ID                 CLUSTER           SUBSYSTEM   TYPE   CR TYPE   AGE
mgmt-backup-8hqqg   Ready    20200826-161301F   m1-dc1-postgres   m1          full   record    127m
mgmt-backup-fbz6w   Ready    20200826-162422F   m1-dc1-postgres   m1          full   record    116m
mgmt-backup-llhzq   Ready    20200826-154646F   m1-dc1-postgres   m1          full   record    153m
mgmt-backup-rwgp2   Ready    20200826-160010F   m1-dc1-postgres   m1          full   record    140m
mgmt-backup-tfj95   Ready    20200826-152658F   m1-dc1-postgres   m1          full   record    173m

$ ls -l /backrestrepo/m1-dc1-postgres-backrest-shared-repo/backup/db/
total 8
drwxr-s--- 3 pgbackrest 26   72 Aug 26 15:27 20200826-152658F
drwxr-s--- 3 pgbackrest 26   72 Aug 26 15:47 20200826-154646F
drwxr-s--- 3 pgbackrest 26   72 Aug 26 16:00 20200826-160010F
drwxr-s--- 3 pgbackrest 26   72 Aug 26 16:13 20200826-161301F
drwxr-s--- 3 pgbackrest 26   72 Aug 26 16:24 20200826-162422F
drwxr-s--- 3 pgbackrest 26   18 Aug 26 15:27 backup.history
-rw-r----- 1 pgbackrest 26 3288 Aug 26 16:24 backup.info
-rw-r----- 1 pgbackrest 26 3288 Aug 26 16:24 backup.info.copy
lrwxrwxrwx 1 pgbackrest 26   16 Aug 26 16:24 latest -> 20200826-162422F

In this example, the file stored on the sftp server is 20200826-162422F.tar.

The script also has the ability to list the backup files available on the sftp server.

When the script is invoked to download a selected file, the contents of the /backrestrepo directory in the Backup and Restore Pod are replaced with the contents of the backup file. A ManagementRestore custom resource can be created, pointing to the relevant backup name that relates to the backup ID, in order to perform a restore. For example:

$ cat mgmtrestore_cr.yaml
apiVersion: management.apiconnect.ibm.com/v1beta1
kind: ManagementRestore
metadata:
  generateName: mgmt-restore-
spec:
  backupName: mgmt-backup-fbz6w

$ kubectl create -f mgmtrestore_cr.yaml

The sftp backup script has the following parameters:

$ /sftp/backup.sh —help
usage: backup.sh ACTION OPTIONS...
  ACTION:
    [-L|--list]     list backup files available remotely
    [-D|--download] download backup file from remote system
    [-U|--upload]   upload backup file to remote system (default)
  OPTIONS:
     -H|--host      host to upload backup to
    [-P|--port]     port for upload protocol. (default 22)
     -d|--DIRECTORY       directory to upload to
     -u|--username  username to use for upload
    [-p|--password] password to use for upload
    [-k|--key]      ssh key to use for upload
    [-r|--retries]  number of retries if there is a failure
    [-n|--name]     name of the backup compressed file (default last backup-id)
    [-h|--help]     help