IBM Support

QRadar: Troubleshooting performance with dumpDSMInfo.sh

Troubleshooting


Problem

Troubleshooting
 
Problem
When DSMs or custom properties are unoptimized, QRadar performance may degrade, resulting in warnings like "Device Parsing has sent a total of X event(s) directly to storage" in logs. This guide details how to use the dumpDSMInfo.sh script to isolate and resolve parsing and normalization issues with DSMs and custom event properties.

Cause

Performance issues typically arise from:
  • Inefficient regular expressions (regex) in DSM extensions or custom properties, which use excessive processing power during event parsing.
  • Recently installed/modified DSMs or properties that introduce new, complex logic.
  • Large event volumes or high event rates, which exacerbate the impact of less optimized DSMs.

Such issues can lead to events being dropped or routed directly to storage, impacting log fidelity and system performance

What are 

Unrecognised or unknown events in QRadar indicate that the log was collected successfully, but the DSM could not parse or categorize it. QRadar still receives the event, but it cannot map it to a known log source or event type, so it is shown as “Unknown” in Log Activity.

Difference between unknown and stored events
Parsed events are the logs that QRadar has successfully categorized into QID, category, and other fields. Unknown events are logs QRadar received and partially parsed, but it could not match them to a known event type or log source. Stored events are logs that QRadar saves to disk without parsing them at all.

Using DSM Editor to map these events
You can open the unrecognised event in the DSM Editor and check the payload. From there, you can create your own regex patterns to extract key fields and map them to a custom QID. Once saved, QRadar will classify similar events correctly in the future.

For reference, here is a link that shows an example for Linux events, which can be adapted to any log source type:
https://www.ibm.com/support/pages/qradar-how-find-non-linux-os-events-getting-linux-log-sources

Diagnosing The Problem

 
You will see similar message  in qradar.error like:
[Timer-86] com.ibm.si.ec.filters.normalize.DSMFilter: [WARN] [NOT:000000000][xxx.xx.xx.xxx/- -] [-/- -]Expensive Log Source or Log Source Extensions Based On Average Throughput in the last 60 seconds (most to least expensive) - DSM A=182.0eps, DSM B=1160.0eps, DSM C=1426.0eps, DSM D=1606.0eps, DSM E=2050.0eps, DSM F=2420.0eps, DSM G=2913.0eps, DSM H=2945.0eps
 

When you see this message in qradar.error you will know that there are expensive DSM in your system. This message will also show you names of DSMs and their EPS rate. In this example DSM A, DSM B , DSM C and so on is used but you will see actual names of DSM.

Resolving The Problem

 
What is the dumpDSMInfo script?
The dumpDSMInfo.sh script in IBM QRadar is a support tool designed to collect and display detailed system level statistics related to event collection, parsing, and normalization for all active Device Support Modules (DSMs). Its primary purpose is to help identify parsing and normalization performance issues caused by DSMs that may be inefficiently processing events. By running this script, administrators can get insights such as how long it takes to process DSM, total execution times, and average execution times. This information is crucial for diagnosing and troubleshooting performance degradation related to DSM processing in the QRadar event pipeline. When the script completes, it turns off these performance metrics.
 
Part 1: How to run dumpDSMInfo.sh script?:
Depending on whether you have QRADAR SIEM or QRADAR SAAS use one of these two methods:

 
 

Generating the report on QRadar SIEM

1. Access the Console CLI:-
  • Log in via SSH to the QRadar system experiencing degraded performance.
2. Run the dumpDSMInfo.sh Script
/opt/qradar/support/dumpDSMInfo.sh
 
  • This script provides statistics on event parsing, DSM resource consumption, normalized vs dropped event counts, and CPU time per DSM.
You will see the file as follows:-
Beans are in ecs-mbeans.tgz
 
3.  Use WinSCP or an equivalent tool to move the ecs-mbeans.tz file to your local laptop or workstation.
4. Use a compression utility to extract the ecs-mbeans.tgz file to a .tar file.
5. Extract the .tar file a second time to access the Expensive DSM report text file. The output contains many .txt file
6. Open dsm.txt file in any spreadsheet program as a CSV file.
 
Alternatively the output can be analyzed by command line by:-
  • Decompressing the file generated
tar -xvzf <file_name>
Example:
tar -xvzf ecs-mbeans.tgz
  • Moving to the new decompressed folder:
cd DSM-2025-10-25-17-37
  • Open the dsm.txt file
head dsm.txt
 
 

Generating the report on QRadar SAAS

  1. To generate the performance report, follow the steps from this document. Select the desired managed host and the  dumpDSMInfo.sh script.
  2. With the downloaded file, use a compression utility to extract the ecs-mbeans.tgz file to a .tar file.
  3. Extract the .tar file a second time to access the Expensive DSM report text file. The output contains a .txt file, .xml file, and a folder named "reports".
  4. Open dsm.txt file in any spreadsheet program as a CSV file.
Moving to the new decompressed folder:

  1. Decompressing the file generate:

    tar -xvzf <file_name>
Example:
tar -xvzf DSM-2025-10-25-17-37.tgz
     

  2. Moving to the new decompressed folder:

    cd DSM-2025-10-25-17-37
     

  3. Open the dsm.txt file

    (echo -e "MBean\tAverageEventParseTimeInMilliSecs"; awk -F'\t' 'NR>1 && $2!="NaN" {print $1, $2}' OFS='\t' ecs-mbeans/dsm.txt | sort -t $'\t' -k2,2nr)
     
 
 
Part 2: What to look for in expensive dsm.txt file?
 
  • Filter the AverageEventParseTimeInMilliSecs in descending order to look for larger values. It identifies which DSM takes more time to execute than others. Expensive DSM are listed first and are typically with large values, you can remove NaN from filter selection. Look for values greater than 0.1 or larger which are considered potentially expensive and require review.
  • Alternatively, you can also review TotalEventParseTimeInMilliSecs to find DSMs that are taking more time to parse than others.
  • Sort the table by amount of unrecognised events and compare with EventsReceived. If unrecognized is over 5%, check unrecognised events section. 
Typically DSMs that have auto-detection on are more expensive.
 
image-20250911151543-1

Output in CLI using command:-
(echo -e "MBean\tAverageEventParseTimeInMilliSecs"; awk -F'\t' 'NR>1 && $2!="NaN" {print $1, $2}' OFS='\t' ecs-mbeans/dsm.txt | sort -t $'\t' -k2,2nr)
 
image-20250911151639-2
 
Part 3: What to do next?
Select one of the following options:
Sep 23 12:32:53 ::ffff:10.1.2.4 [ecs-ec] 
[Timer-57] com.q1labs.semsources.filters.normalize.DSMFilter: [WARN]  
[NOT:0080014100][10.1.2.4/- -][-/- -]Expensive Log Source or Log Source 
Extensions Based On Average Throughput in the last 60 seconds 
(most to least expensive) - Checkpoint=0.0eps, CatOS=86.0eps, Apache=2500.0eps,
Endpointprotection=2905.0eps
 
  1. Disable any DSM extension that was recently installed.
  2. Review the payload of the notification to determine which expensive DSM extension in the pipeline affects performance. If possible, improve the regex statements that are associated with the device extension.                                                                                                             For example, the following payload reports that the pipeline is blocked by the Checkpoint DSM:

  3. Ensure that the log source extension is applied only to the correct log sources.

    On the Admin tab, click System Configuration > Data Sources > Log Sources. Select each log source and click Edit to verify the log source details.

  4. If you are working with protocol-based log sources, reduce the event throttle to ensure that the events do not buffer to disk. The event throttle settings are part of the protocol configuration for the log source.
  5. Order your log source parsers from the log sources with the most sent events to the least and disable unused parsers.

  6. Verify that your Console is installed with the latest DSM versions.

    rpm -qa |grep dsm
    Verify this with DSM version published on Fix Central
  7. Check the DSMs are supported or not:-https://www.ibm.com/docs/en/dsm?topic=configuration-qradar-supported-dsms
  8. If log sources are created for devices that aren’t in your environment, disable auto detection:- DSM editor ->Choose DSM 
  9. Deploy Full Configuration

 
 

[{"Type":"MASTER","Line of Business":{"code":"LOB77","label":"Automation Platform"},"Business Unit":{"code":"BU048","label":"IBM Software"},"Product":{"code":"SSV4BL","label":"IBM QRadar"},"ARM Category":[{"code":"a8m0z000000cwtiAAA","label":"Performance"}],"ARM Case Number":"","Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"7.3.0;7.4.0;7.5.0"}]

Document Information

Modified date:
18 November 2025

UID

ibm17244028

Page Feedback