Offline backup fails with error pruning repository error
Creating an offline backup with the OADP utility fails with the error message error pruning repository.
Symptoms
In the CPD-CLI*.log file, you see error messages like in the following
example:
time=<timestamp> level=info msg=velero ns: oadp-operator
time=<timestamp> level=error msg=Backup "xxxxxx" does not exist.
time="<timestamp>" level=error msg=cpd-cli error="RunPluginCommand:Execution error: exit status 1" event=Exc
In the Velero log, you see an error message like in the following
example:
time="<timestamp>" level=warning msg="error pruning repository" error="error running command=restic prune
In the velero-xxxx-xxxxx/velero-xxxx-xxxxx.yaml from the OADP
must-gather tool, you see an
OOMKilled error like in the
following
example:- containerID: cri-o://b36031fce37cdd42dc0bf7b4c96997b33d2745c6f754e7c1419632229bf57e54
image: registry.redhat.io/oadp/oadp-velero-rhel8@sha256:2c1c645410a77569705e5521f8abb652b2b46bcb6ee06b9650bde78fcf3e3061
imageID: registry.redhat.io/oadp/oadp-velero-rhel8@sha256:2c1c645410a77569705e5521f8abb652b2b46bcb6ee06b9650bde78fcf3e3061
lastState:
terminated:
containerID: cri-o://3bf3b24d6b6375a6526866fbf8e452bf457f8aa010c729ce1f0ad18059a5dffe
exitCode: 137
finishedAt: "2023-10-02T15:48:16Z"
reason: OOMKilled
startedAt: "<timestamp>"
name: velero
Causes
An OOMKilled error occurred in the Velero pod.
Diagnosing the problem
Collect log information with the oadp gather info command.
Resolving the problem
In the DataProtectionApplication (DPA), increase the Velero memory limit.
- Edit the
DPA:
oc edit dpa -n ${OADP_OPERATOR_PROJECT} - Update the memory limit in the following places:
spec.configuration.velero.podConfig.resourceAllocations.limits.memoryspec.configuration.nodeAgent.podConfig.resourceAllocations.limits.memory
For example:
spec:
configuration:
velero:
customPlugins:
- image: ${CPDBR_VELERO_PLUGIN_IMAGE_LOCATION}
name: cpdbr-velero-plugin
defaultPlugins:
- aws
- openshift
- csi
podConfig:
resourceAllocations:
limits:
cpu: "${VELERO_POD_CPU_LIMIT}"
memory: 4Gi
requests:
cpu: 500m
memory: 256Mi
resourceTimeout: 60m
nodeAgent:
enable: true
uploaderType: restic
timeout: 72h
podConfig:
resourceAllocations:
limits:
cpu: "${NODE_AGENT_POD_CPU_LIMIT}"
memory: 32Gi
requests:
cpu: 500m
memory: 256Mi
tolerations:
- key: icp4data
operator: Exists
effect: NoSchedule