Troubleshooting
Problem
Cause
Diagnosing The Problem
- Complete any search, add a filter by Event Processor, and from the View, list select Real Time (streaming).
If you do not see any streaming events, this can indicate an issue with the ecs-ep process, which manages real-time event streaming from the Event Processor appliance to the Console. - To verify that the ecs-ep process is running from the command line of the QRadar appliance, enter:
service ecs-ep status
If the service reports asecs-ep is stopped
the administrator should attempt to restart the process, using the following command:
service ecs-ep start
- The most frequent cause of processes not running on configured systems in a deployment is due to the disk space issue. If this occurs, a system notification will be generated to alert the administrator to the issue.
[root@qradar119 ~]# grep -i "disk usage" /var/log/qradar.error
Nov 4 06:52:11 ::ffff:IP Address [hostcontext.hostcontext] [c0ac7072-70e9-40ea-9d87-62ac50d090c3/SequentialEventDispatcher] com.q1labs.hostcontext.ds.DiskSpaceSentinel: [ERROR] [NOT:0150064100][IP Address/- -] [-/- -]Disk usage on at least one disk has exceeded the maximum threshold level of 0.95. The following disks have exceeded the maximum threshold level: /store, Processes are being shut down to prevent data corruption. To minimize the disruption in service, reduce disk usage on this system.
-
Other ways verify disk space on your appliances.
To view disk usage:
In QRadar 7.2.8
# df -Th
Filesystem Type Size Used Avail Use% Mounted on
/dev/sda7 ext4 20G 7.1G 12G 38% /
tmpfs tmpfs 12G 0 12G 0% /dev/shm
/dev/sda1 ext4 97M 39M 53M 43% /boot
/dev/sda8 xfs 30G 29G 2.0G 95% /store
/dev/sda6 ext4 9.8G 152M 9.2G 2% /store/tmp
/dev/sda9 xfs 7.7G 6.3G 1.4G 82% /store/transient
/dev/sda5 ext4 9.9G 463M 8.9G 5% /var/log
In QRadar 7.3
[root@QRadar732 support]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/rootrhel-root 13G 5.3G 7.3G 43% /
devtmpfs 16G 0 16G 0% /dev
tmpfs 16G 20K 16G 1% /dev/shm
tmpfs 16G 34M 16G 1% /run
tmpfs 16G 0 16G 0% /sys/fs/cgroup
/dev/sda3 32G 4.1G 28G 13% /recovery
/dev/sda2 1014M 163M 852M 17% /boot
/dev/mapper/rootrhel-opt 13G 2.7G 9.9G 22% /opt
/dev/mapper/rootrhel-tmp 3.0G 41M 3.0G 2% /tmp
/dev/mapper/rootrhel-var 5.0G 175M 4.9G 4% /var
/dev/mapper/rootrhel-home 1014M 33M 982M 4% /home
/dev/mapper/storerhel-store 142G 33G 109G 24% /store
/dev/mapper/rootrhel-varlog 15G 387M 15G 3% /var/log
/dev/mapper/rootrhel-storetmp 15G 43M 15G 1% /storetmp
/dev/mapper/rootrhel-varlogaudit 3.0G 131M 2.9G 5% /var/log/audit
/dev/mapper/storerhel-transient 36G 36M 36G 1% /transient
tmpfs 3.1G 0 3.1G 0% /run/user/0
To get the disk usage on all your appliances, use the following command:
/opt/qradar/support/all_servers.sh -C -k "df -Th"
-
Lastly, you can run /opt/qradar/support/deployment_info.sh. This script will collect all information about all systems in the deployment including disk space used, hardware, appliance type and serial number within a CSV file.
Resolving The Problem
When you free up space on the appliance, services will automatically restart when the system detects that disk usage is below 92%.
Where do you find more information?







Was this topic helpful?
Document Information
Modified date:
02 May 2019
UID
swg21690477