How To
Summary
How to check the current capacity of the DSM Normalize Queue or the Custom Rules Engine (CRE) Queue on a QRadar managed host.
Objective
Each step in the QRadar event pipeline leverages queues for events to await the resources required to parse events or check the events against the rules to create offenses. As system load increases and decreases, the values of how many events waiting to be parsed or checked by the Rules Engine might also increase or decrease.
Custom Rule Engine has sent a total of 2209032 event(s) directly to storage. 242592 event(s) were sent in the last 60 seconds. Queue is at 99 percent capacity.
This message occurring in the QRadar application logs or as a System Notification as an indicator that the Custom Rule Engine queue is close to capacity.
Note: It is advisable to check the immediate status of both the DSM Normalize Queue and Custom Rule Engine queues.
Steps
Using ssh, connect to your QRadar® console. Depending on which managed host you want to check, you can continue to ssh to that host.
To check the queue status of event parsing
Run the following command:
/opt/qradar/support/jmx.sh -p 7777 -b 'com.q1labs.sem:application=ecs-ec.ecs-ec,type=filters,name=DSM'
Note the following output:
In this case, the ParserQueueSize is equal to 0. This value indicates the queue is empty. The ParserMaxQueueSize indicates the maximum number of events that can be waiting to be parsed before the system starts dropping events. In this case, it's 75000.
To check the queue status of the CRE queue
Run the following command:
Run the following command:
/opt/qradar/support/jmx.sh -p 7799 -b 'com.q1labs.sem:application=ecs-ep.ecs-ep,type=filters,name=CRE'
Note the QueueSize is 0 in this case. The QueueCapacity shows the maximum number of events that can be waiting in the CRE queue before it starts to drop events. In this case, the value is 50000.
If the output of these commands is No matching mbeans that can mean that the command was incorrectly input or that the MBean is not exposed (for example, during a restart of services).
Additionally, queues that are available on a particular host depends on the functionality of the host. For example, the command for the Custom Rule Engine queue returns No matching mbeans if run on a dedicated Event Collector (EC), as ECs are not designed to process rules.
Likewise, if you run the DSM Normalize command on a dedicated Event Processor (EP) it returns No matching mbeans as EPs have no Event Parsing capability.
A useful technique for checking queues is to combine the DSM Normalize and CRE commands and the watch command to monitor them in real time, for example:
watch -n 1 "/opt/qradar/support/jmx.sh -p 7799 -b 'com.q1labs.sem:application=ecs-ep.ecs-ep,type=filters,name=CRE'"
This command checks the status of the queue once per second until stopped.
Additional Information
Document Location
Worldwide
[{"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SSBQAC","label":"IBM Security QRadar SIEM"},"Component":"","Platform":[{"code":"PF016","label":"Linux"}],"Version":"7.3.x","Edition":"","Line of Business":{"code":"LOB24","label":"Security Software"}}]
Was this topic helpful?
Document Information
Modified date:
24 November 2020
UID
ibm13130677