IBM Support

QRadar: Nighly backups fail to run with "Unable to determine available disk space, aborting backup" error

Troubleshooting


Problem

Nightly backups fail to run when a remote mount is not reachable or not readable. 

Warning: If you use NFS or a Windows share for offboard storage, your system can lock and cause an outage. This practice is not supported by IBM QRadar. If you choose to use NFS anyway, NFS can be used only for daily backup data, such as the /store/backup directory. You cannot use NFS for storing active data, which includes the PostgreSQL and ariel databases. If you do use NFS, it might cause database corruption or performance issues.

Symptom

A system notification is received for "Unable to execute backup request" on the affected managed host.

Figure01

Environment

QRadar Deployments with remote mounts.

Diagnosing The Problem

Connect to the affected host, review the qradar.error to determine the affected remote partition.
 
  1. Use SSH to log in to the QRadar Console as the root user.
  2. Optional. SSH to the affected managed host.
  3. Review the /var/log/qradar.log and confirm the presence of the error described in the Symptom section: 
    grep -i "Unable to determine available disk space" /var/log/qradar.log
    Output example: 
    [hostcontext.hostcontext] [Scheduled Backup] com.q1labs.hostcontext.backup.BackupRecoveryEngine:
 [ERROR] [NOT:0000003000][xx.xx.xx.xx/- -] [-/- -]Unable to determine available disk space, aborting backup
  4. Use the following command to confirm the use of remote mounts: 
    mount -v
    Note: Remote mounts are displayed with an IP or FQDN.

    Result
    External storage mount points are displayed. 

Resolving The Problem

Administrators are encouraged to read the Offboard storage overview and Backup and recovery sections of the QRadar Administration Guide. 

To resolve the problem, unmount the conflicting remote partition and test taking an on-demand backup.

  1. Use SSH to log in to the QRadar Console as the root user.
  2. Optional. SSH to the affected managed host.
  3. Test the availability of the remote directories.  All commands must succeed. 
    cd /path_to_remote_directory/ 
touch test.txt
echo "remote test" > test.txt 
ls test.txt
rm -fv test.txt
    Output example: 
    [root@hostname~]# cd /store/backup-test/
[root@hostname backup-test]# touch test.txt
[root@hostname backup-test]# echo "remote test" > test.txt
[root@hostname backup-test]# ls test.txt
test.txt
[root@hostname backup-test]# rm -fv test.txt
removed 'test.txt'
  4.  Unmount all remote mount points that failed in step 3. Change the following command for each remote mount: 
    umount -f -l <path_to_remote_directory>
  5. Take an on-demand backup. For detailed instructions to do so, see Creating an on-demand configuration backup archive.
  6. Verify the on-demand backup file is stored in /store/backup.

    Result
    The on-demand configuration backup completes successfully. Before mounting the remote directories again, administrators must ensure only supported remote storage is configured and reachable.

    If the backup continues to fail with the same error contact QRadar Support for assistance.

Document Location

Worldwide

[{"Type":"MASTER","Line of Business":{"code":"LOB24","label":"Security Software"},"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SSBQAC","label":"IBM Security QRadar SIEM"},"ARM Category":[{"code":"a8m0z000000cwtNAAQ","label":"Deployment"}],"ARM Case Number":"","Platform":[{"code":"PF016","label":"Linux"}],"Version":"All Versions"}]

Document Information

Modified date:
30 November 2022

UID

ibm16841513

Page Feedback