Troubleshooting
Problem
Are there any tips to improve search efficiency in QRadar?
Resolving The Problem
Tip #1: Narrow Your Search Scope
The more you can tell QRadar about the data you are looking for, the faster QRadar will return results. A targeted search is much more efficient and any additional indexed values that can be added will help target the data that your users require.
The most important scope that can be narrowed is time. There is a nearly linear relationship between the time range you search and the length of time it takes results to return. For example, two searches are started using the same search criteria. One search is evaluating data over the last 120 days and the second search is evaluating data over the last 60 days. On average, the 120 day search will take roughly twice as long as searching 60 days of data.
When starting a search, users can improve the search efficiency by specifying a shortened time window. After the time interval for the search is defined, the search scope can be narrowed further by adding indexed properties to your search. As an example, if you know that all of your firewall devices are assigned a certain address range, then add that range to the search using the Source or Destination Network filter, in addition to the firewall logs you are looking for.
Tip #2: Start Small and Grow the Your Search
One common issue that occurs when searching is that users know the data they are looking for, but not exactly when the data or incident occurred. A good example of this is investigating an event from a specific IP address from a wide time range. The user knows that the data exists, the IP address that needs to be investigated, and that data in within a 6 month window. A normal response to this scenario is to search the past six months of logs for the IP address; however, in many cases this is not the most efficient method to locate data and can consume time or resources unnecessarily. The recommended best practice when trying to search a long time period, is to start with a narrow time period, and gradually widen it.
For example, you may want to search only the past week, then widen the search to two weeks, then widen it to four weeks, then two months, then six months. During each of these search cycles, you may decide that you have everything you need and don't need to widen it any more, or you may find more and more information to use in your search. The information learned during progressive searches can then be used as filters for when you grow your search out to a 6 month window. The additional filters added might end up with a targeted six month query run faster than a generic two week search!
Another reason to start a small search and grow, is due to how distributed searches work. Searches always result in the creation of a database cursor on the Console. The purposes of these cursors is to allow for additional filters that might be added to a search result and provides information for rapid resorting and data aggregation. Therefore, the larger the size of the initial search period (which is likely less targeted than your final search will be), the more search results you have to bring back over the network to the Console. For large distributed deployments of appliances, sometimes the network bandwidth of bringing the results back to the Console can have just as much of an impact on performance as the disk I/O of the search itself. This is especially true when low bandwidth or high-latency connections are part of the deployment picture. By starting small and only widening the search query as needed, you are limiting the amount of data brought back to the Console to only what you actually need in your investigation.
Tip #3 – Isolate Your Search To A Specific Event or Flow Processor
There are two search filters in QRadar that behave in a special way: the Event Processor and Flow Processor filter. These filters do not utilize an index to narrow the data, but instead tell the QRadar Console to run the search against the data on a specific appliance. This can be helpful if you know what appliance manages what data. When you know in advance where the data resides in your QRadar deployment, these filters can be used to dramatic effect.
Figure 1: The Event Processor filter displays a list of appliances that can process events.
Note: There is a separate filter for Flow Processor to help target searches.
For example, a QRadar deployment contains two event processors, one appliance handles internal logging for events within the local network (such as database auditing, Windows events, internal antivirus events) and another appliance that handles events for perimeter logging (Internet border and DMZ event data) . When you start your search, knowing in advance that you are looking for external log data allows users to filter by Event Processor to run targeted searches. The Event Processor filter tells QRadar that it can ignore the internal logging appliance and directs searches to the correct appliance.
Related Information
[{"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SSBQAC","label":"IBM Security QRadar SIEM"},"ARM Category":[{"code":"a8m0z000000GnblAAC","label":"QRadar->Search"}],"ARM Case Number":"","Platform":[{"code":"PF016","label":"Linux"}],"Version":"7.2;7.3;7.4","Edition":"All Editions","Line of Business":{"code":"LOB24","label":"Security Software"}}]
Was this topic helpful?
Document Information
Modified date:
02 April 2020
UID
swg21691783