About cookies on this site Our websites require some cookies to function properly (required). In addition, other cookies may be used with your consent to analyze site usage, improve the user experience and for advertising. For more information, please review your options. By visiting our website, you agree to our processing of information as described in IBM’sprivacy statement. To provide a smooth navigation, your cookie preferences will be shared across the IBM web domains listed here.
White Papers
Abstract
This document outlines factors that contribute to Device Parsing Performance Degradation on a QRadar environment.
Content
Custom Property performance issues in QRadar are most commonly noted in the form of performance degradation messages, such as the following:
[ecs-ec.ecs-ec] [[type=com.eventgnosis.system.ThreadedEventProcessor][parent=qradar.local:ecs-ec/EC/Parsing/DSM_Normalize]] com.ibm.si.ec.filters.normalize.DSMFilter: [WARN] [NOT:0080000001][172.1.2.3/- -] [-/- -]Device Parsing has sent a total of 16731904 event(s) directly to storage. 50540 event(s) have been sent in the last 60 seconds. Queue is at 99 percent capacityQRadar parses each event that it receives against each enabled custom property configured for it is log source on the system. The maximum Events per Second that can be processed by the event collection service depends on the host's hardware specifications, the kind of events it is ingesting and the complexity of the custom properties that are defined. If this threshold is breached, the system sends the events it was not able to process directly to storage, bypassing full evaluation of the event parsing service. This behaviour is referred to as Performance Degradation.
The maximum thresholds for Events Per Second rates for QRadar hosts are detailed in the system specifications and are based on a defined certification methodology.
How a host performs when parsing properties depends on a number of factors:
- Number of cores
- Number of parsing threads
- Nature of the EPS
- Number of properties
- Complexity of properties
Number of CPU Cores and Threads
These two factors are the physical limitations of the host, and are fairly simple; the greater the number of CPU cores and parsing threads, the faster the host can parse the enabled properties, and the higher the amount of EPS a host can parse.
The IBM Guidelines give an indication of the required number of cores based the different thresholds of EPS in the following link:
It is worth noting these specifications are based on a test set of properties, the specifics of which can be found in the following link:
https://www.ibm.com/docs/en/qsip/7.5?topic=overview-qradar-maximum-eps-certification-methodology
As a result, depending on how a system is used, the maximum load will differ. For example a very expensive and large set of properties will decrease the amount of EPS a host can parse.
The number of parsing threads by default is roughly equal to 40% of the number of CPU cores on an Event Processor, and 75% on an Event Collector. If the number of cores is increased, the best practice is to reboot the host, to ensure the thread count is updates in line with the core count.
EPS Nature, Number of Properties and Property Complexity
The EPS Nature, and complexity and number of the properties all refer to the load on the system caused by parsing the events. The system parses the number of events it receives in each second against the enabled property on the system.
There are three basic conclusions we can draw from this example:
- The more complex a property is, the longer time to parse it.
- The more properties there are, the longer total time to parse each event.
- The more events there are, the longer total time to parse them all.
These three factors contribute to create a maximum EPS threshold that the host can parse in a single second. If this threshold is breached, in order to make sure it does not fall behind, the system sends some events directly to storage (Performance degradation), and starts parsing the next second's events.
EPS Nature
Both the number and complexity of the properties on a system can affect the maximum number of events a system can parse. Every property added to a system will have a cost to run it, which is an expression of time it takes to parse that property. The longer the total time to parse the properties, the lower the maximum threshold of EPS the host con parse. If a spike in the EPS takes the system past the threshold of what it can parse, then performance degradation occurs. This is why performance degradation is often seen during the working day, when the most systems see an increase in the number of events they have to parse.
If a host is receiving a high level of events per second passing through the system e.g. 20,000+ EPS, it is doubly important to have efficient properties. With a high EPS, the inefficiencies of any property are maximised. For example, if a property takes on average 0.001 milliseconds to parse against 1 event, and the system is receiving 1000 eps, then this property will take a total 1 second to parse. If the same host receives 25,000 EPS, this property will take 25 seconds to parse.
Number of properties
The number of properties a system can support depends heavily on each of the other described factors. Each time a property is added to a system, it slightly increases the number of tests the system has to test against each event received, and so increases the property parsing load. Every property added therefore increases the length of time the parsing service needs to parse each event, and thus decreases the maximum amount of EPS the host can handle. If the EPS rate is low, or the property tests are efficient, then the maximum number of properties a system can support and successfully parse is higher.
Property Complexity
Each property runs a number of comparisons to detect whether there is a match. The higher the number of comparisons run, the more complex the property. In each property, the aim is to keep the number of comparisons as low as possible, to keep the property efficient.
Most property tests are made of two components:
- The data we want to detect (Called the pattern)
- The data in the event we want to search (Called the string)
The pattern is the search the property runs. In most cases this will be either regex or AQL.
The string is the event payload.
Additional notes:
The longer the average payload, the greater the average number of comparisons run to find the data to extract.
The more options the regex has, the more comparisons it will need to do to find the data.
If a property finds a match, it will stop searching the string. this is why all properties should try to extract in as few comparisons as possible. If however there is a property that cannot find a match, it will search the entire string to try to find one, thus making the property more expensive.
Related Information
[{"Type":"MASTER","Line of Business":{"code":"LOB24","label":"Security Software"},"Business Unit":{"code":"BU048","label":"IBM Software"},"Product":{"code":"SSV4BL","label":"IBM QRadar"},"ARM Category":[{"code":"a8m0z000000cwtiAAA","label":"Performance"}],"Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"All Versions"}]
Was this topic helpful?
Document Information
More support for:
IBM QRadar
Component:
Performance
Software version:
All Versions
Document number:
7179379
Modified date:
09 January 2025
UID
ibm17179379
Manage My Notification Subscriptions