QRadar: How to identify disk space issues in the root partition / by excluding mount points

Steps

1. SSH into your QRadar console.
2. Run the following command and observe the output:

   df -h /

   In this example, we can see / is using 95% of its disk space.

   df -Th
   Filesystem                       Type      Size  Used Avail Use% Mounted on
   devtmpfs                         devtmpfs   16G     0   16G   0% /dev
   tmpfs                            tmpfs      16G  904K   16G   1% /dev/shm
   tmpfs                            tmpfs      16G   26M   16G   1% /run
   tmpfs                            tmpfs      16G     0   16G   0% /sys/fs/cgroup
   /dev/mapper/rootrhel-root        xfs        13G  4.9G  7.7G  95% /
   /dev/mapper/rootrhel-home        xfs      1014M   35M  980M   4% /home
   /dev/mapper/rootrhel-storetmp    xfs        15G  4.6G   11G  31% /storetmp
   /dev/sda3                        xfs        32G  5.4G   27G  17% /recovery
   /dev/sda2                        xfs      1014M  354M  661M  35% /boot
   /dev/mapper/rootrhel-var         xfs       5.0G  284M  4.8G   6% /var
   /dev/mapper/rootrhel-opt         xfs        13G  4.3G  8.3G  34% /opt
   /dev/mapper/storerhel-transient  xfs        36G   39M   36G   1% /transient
   /dev/mapper/rootrhel-tmp         xfs       3.0G   67M  3.0G   3% /tmp
   /dev/mapper/rootrhel-varlog      xfs        15G  1.4G   14G  10% /var/log
   /dev/mapper/rootrhel-varlogaudit xfs       3.0G   96M  2.9G   4% /var/log/audit
   /dev/drbd0                       xfs       142G   36G  107G  25% /store
   tmpfs                            tmpfs     3.2G     0  3.2G   0% /run/user/0

3. If you run du -hs *, it shows the space used in each directory, however this scans every mount point, which can run slowly on large deployments. To work around this issue, you can mount the / partition to a temporary location and then run the commands there. This step prevents the scanning of external mount points. The following commands are an example of mounting the root partition to /media/cdrom:

   mount -o bind / /media/cdrom

4. You can now run du -hs * in the mounted directory to see all the files in / excluding the mount points like /store, /transient, etc.

   cd /media/cdrom
   du -hs *

   In the example, the /test directory is using the largest amount of space:

   0	bin
   0	boot
   0	dev
   42M	etc
   0	home
   0	lib
   0	lib64
   0	media
   0	mnt
   0	opt
   0	proc
   0	recovery
   172K	root
   8.0K	run
   0	sbin
   0	srv
   4.0K	store
   0	storetmp
   0	sys
   7.1G	test
   0	tmp
   0	transient
   3.4G	usr
   0	var
   

5. After you remove the files that use too much space, you can check again by running the following command:

   df -Th

   Example output:

   Filesystem                       Type      Size  Used Avail Use% Mounted on
   devtmpfs                         devtmpfs   16G     0   16G   0% /dev
   tmpfs                            tmpfs      16G  120K   16G   1% /dev/shm
   tmpfs                            tmpfs      16G   26M   16G   1% /run
   tmpfs                            tmpfs      16G     0   16G   0% /sys/fs/cgroup
   /dev/mapper/rootrhel-root        xfs        13G  4.9G  7.7G  39% /
   /dev/mapper/rootrhel-home        xfs      1014M   35M  980M   4% /home
   /dev/mapper/rootrhel-storetmp    xfs        15G  201M   15G   2% /storetmp
   /dev/sda3                        xfs        32G  5.4G   27G  17% /recovery
   /dev/sda2                        xfs      1014M  354M  661M  35% /boot
   /dev/mapper/rootrhel-var         xfs       5.0G  286M  4.8G   6% /var
   /dev/mapper/rootrhel-opt         xfs        13G  4.3G  8.3G  34% /opt
   /dev/mapper/storerhel-transient  xfs        36G   39M   36G   1% /transient
   /dev/mapper/rootrhel-tmp         xfs       3.0G   67M  3.0G   3% /tmp
   /dev/mapper/rootrhel-varlog      xfs        15G  983M   15G   7% /var/log
   /dev/mapper/rootrhel-varlogaudit xfs       3.0G   88M  3.0G   3% /var/log/audit
   /dev/drbd0                       xfs       142G   35G  107G  25% /store
   tmpfs                            tmpfs     3.2G     0  3.2G   0% /run/user/0
   

6. After you complete the cleanup, you can safely unmount the temporary mount point:

   cd /
   umount /media/cdrom

   
   Result
   After you identify the directories and files that are taking up too much space and clean them, you can run the following command to ensure that /media/cdrom is no longer mounted:

   df -Th