Question & Answer
Question
How is data stored and accessed for searches?
Answer
Event and flow data is stored locally in the /store/ariel/ directory for each QRadar appliance where ECS-EP process runs. The following appliance types include the ECS-EP process:
Searching
Data is written to disk in minute-by-minute intervals and indexed. When a search is performed, the search process (Ariel proxy server) polls remote appliances to determine whether they have results for the time frame defined in the search query. The request is received by the Ariel Query Server on the managed host and any matched results are returned to the Console in the form of a cursor. The cursor is stored locally on the Console until it expires, or until the search is deleted from the Managed Search Results.
As a search runs, users can click More Details to view the search progress on each appliance as they return results.
Real-time data (streaming)
When the Real Time (streaming) view selected from Log Activity or Network Activity, the data is sent in real time to the Console with any search parameters or filters applied.
Network Outages
If an appliance is not accessible or a network outage occurs, then search can mark the appliance as unavailable or N/A. An alert bar is displayed to the user that an appliance was not accessible when appliances are unreachable due to network issues. If the remote appliance has a high-availability secondary, the expected behavior is that the secondary appliance fails over and return the search result. If a failover occurs while searches are running, the administrator might need to wait and retry to search after the standby appliance becomes active and can respond to searches. Optionally, the user can edit the search and add a filter for 'Event Processor is' or 'Flow Processor is' to run the same search against a targeted appliance that previously did not respond. A customized search allows the user to get results from a single appliance without having to query all appliances in the deployment.
- QRadar 16xx Event Processor appliances
- QRadar 17xx Flow Processor appliances
- QRadar 18xx Combination Event and Flow Processor appliances
- QRadar 21xx Log Manager appliances
- QRadar 31xx Consoles
- QRadar 12xx and 13xx QFlow appliances do not store any data locally. The data is sent to a Flow Processor appliance for further evaluation and storage.
- QRadar 15xx Event Collectors receive, parse, and forward events to a QRadar Event Processor as data is received. The QRadar 15xx Event Collector appliance can be configured to temporarily store events and forward the stored events on a schedule; however, this configuration is not common. For more information, see Event store and forward.
Searching
Data is written to disk in minute-by-minute intervals and indexed. When a search is performed, the search process (Ariel proxy server) polls remote appliances to determine whether they have results for the time frame defined in the search query. The request is received by the Ariel Query Server on the managed host and any matched results are returned to the Console in the form of a cursor. The cursor is stored locally on the Console until it expires, or until the search is deleted from the Managed Search Results.
As a search runs, users can click More Details to view the search progress on each appliance as they return results.
Real-time data (streaming)
When the Real Time (streaming) view selected from Log Activity or Network Activity, the data is sent in real time to the Console with any search parameters or filters applied.
Network Outages
If an appliance is not accessible or a network outage occurs, then search can mark the appliance as unavailable or N/A. An alert bar is displayed to the user that an appliance was not accessible when appliances are unreachable due to network issues. If the remote appliance has a high-availability secondary, the expected behavior is that the secondary appliance fails over and return the search result. If a failover occurs while searches are running, the administrator might need to wait and retry to search after the standby appliance becomes active and can respond to searches. Optionally, the user can edit the search and add a filter for 'Event Processor is' or 'Flow Processor is' to run the same search against a targeted appliance that previously did not respond. A customized search allows the user to get results from a single appliance without having to query all appliances in the deployment.
Related Information
[{"Type":"MASTER","Line of Business":{"code":"LOB24","label":"Security Software"},"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SSBQAC","label":"IBM Security QRadar SIEM"},"ARM Category":[{"code":"a8m0z000000cwtEAAQ","label":"Log Activity"}],"ARM Case Number":"","Platform":[{"code":"PF016","label":"Linux"}],"Version":"All Versions"}]
Was this topic helpful?
Document Information
Modified date:
08 June 2021
UID
swg21622708