Troubleshooting
Problem
In QRadar, raw events are ingested and then parsed (normalized) by the ecs-ec service. Within the ecs-ec service, the event parser threads take information from the payload and build a record by using custom event properties and patterns from the respective DSM. If these parser threads become overwhelmed and cannot handle new events as quickly as they arrive at the system, the ecs-ec service routes some events "directly to storage", bypassing the parser threads. This mechanism is designed to preserve as close to real-time processing as possible, but it is important to address the performance issue quickly as unparsed also events impacts correlation and search functionality.
Symptom
The most common way to see Performance Degradation is in the system notifications. In the bell icon a notification that says Performance degradation has been detected in the event pipeline is displayed. If you hover over the notification, the payload specifies which system is experiencing the routing to storage behavior.
In this case, we want to concentrate on routing to storage at DSM filter. This example payload mentions both DSM filter and Device Parsing.
[[type=com.eventgnosis.system.ThreadedEventProcessor][parent=:ecs-ec/EC/Parsing/DSM_Normalize]]
com.ibm.si.ec.filters.normalize.DSMFilter: [WARN] [NOT:0080004101][-/- -]
Device Parsing has sent a total of 18167670 event(s) directly to storage.
593 event(s) have been sent in the last 60 seconds. Queue is at 88 percent capacity.
Document Location
Worldwide
[{"Type":"MASTER","Line of Business":{"code":"LOB77","label":"Automation Platform"},"Business Unit":{"code":"BU048","label":"IBM Software"},"Product":{"code":"SSBQAC","label":"IBM Security QRadar SIEM"},"ARM Category":[{"code":"a8m0z000000cwtiAAA","label":"Performance"}],"ARM Case Number":"","Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"All Versions"}]
Log InLog in to view more of this document
This document has the abstract of a technical article that is available to authorized users once you have logged on. Please use Log in button above to access the full document. After log in, if you do not have the right authorization for this document, there will be instructions on what to do next.
Was this topic helpful?
Document Information
Modified date:
03 April 2023
UID
ibm16603731