News
Abstract
This technical note is an overview from the QRadar Development and Support teams regarding the 20 July 2021 auto update issue for DSM Common, which was reported and verified as a known issue in APAR IJ33892.
Content
Summary
The QRadar auto update released on 20 July 2021 introduced a problem for QRadar 7.4.x users where the Traffic Analysis service that auto discovers and creates log sources is no longer working as expected due to a class loading issue. Users with affected log sources configured on their QRadar managed hosts can experience an uncaught exception with Event Parsers, causing all events to route directly to storage. After an investigation, this issue was caused by a code signing error related to the codegen JAR file within the DSM Common RPM.
Affected QRadar versions
- QRadar SIEM 7.4.x
- QRadar on Cloud
Affected services
| Issue | Summary | Data loss | Symptoms |
|---|---|---|---|
| Traffic Analysis (TA) |
|
No. Events are captured and written to disk. Data is searchable from the Quick Filter or Advanced Search interface to locate stored event data. |
|
| Event Parsers | Systems where an affected log source is enabled experience a ThreadException to occur for the Event Parsers on the managed hosts where events are received. When Event Parser threads enter an error state and cannot restart, the managed host believes it is experiencing a performance issue in the event pipeline and data is written directly to disk to protect the managed host. Affected DSMs:
|
No. Events are captured and all incoming data is written to disk. Data is searchable from the Quick Filter or Advanced Search interface to locate stored event data. |
When events route to storage:
|
Investigation
The following information outlines the analysis completed by QRadar Development teams.
| Category | Action or description |
| File name | DSM Common |
| Version | DSMCommon-7.4-20210624145517.noarch.rpm |
| Severity | Critical |
| Issue type | Auto update for 20 July 2021 (Build 1626784986) |
| Root cause | Code signing / RPM package |
| Issue documentation | |
| Services |
|
| Immediate actions taken | Upon report of the auto update issue, development teams removed affected files from IBM Fix Central and QRadar Auto Update servers on IBM Cloud. A flash notice was issued to all users and support created forum posts to raise visibility of the issue to users. |
| Analysis | DSM Common is a framework file used by QRadar, which includes shared libraries and parsing files used across multiple DSMs that have similar event structures. The DSM Common file for 20 July 2021 included a framework library (q1labs_sem_dsm_codegen.jar) that required a code signing update per IBM's Security Policy. The code signing update changed the class path when the DSM Common RPM file was packaged. The path error caused the NoClassDefFoundError, which then could lead to ThreadException errors for certain DSMs that required the codegen JAR file to parse events. Administrators with log sources enabled from the affected DSM list experienced the ThreadException for the Event Parser, leading to all events being routed to storage. |
| Affected DSMs | The following DSMs reference codegen JAR from DSM Common: Array Networks SSL VPN Access Gateways, Cisco Aironet, CRYPTOCard CRYPTOShield, Extreme HiGuard, Extreme XSR Security Routers, Fair Warning, HP Network Automation, IBM DB2, IBM Informix Audit, Juniper vGW, Juniper Networks AVT, McAfee Application/Change Control, Microsoft ISA, Motorola SymbolAP, Redback ASE, Sentrigo Hedgehog, Silver Springs Networks Smart Meter, Sophos Enterprise Console, Sophos PureMessage, Tropos Control |
| Mitigation | Auto update servers were issued a new version of DSM Common with a codegen JAR file for all QRadar 7.4.x users. Auto update scripts installed the updated codegen.jar from DSM Common to resolve the issue. An updated flash notice was issued to users to "Get New Updates". All users on QRadar 7.4.x received the updated files automatically when the daily auto update completed (approximately 3:00 AM hardware time). Note: Administrators must confirm the Web Server for Auto Updates is configured to use https://auto-update.qradar.ibmcloud.com/. For more information, see the following technical note Important auto update server changes for administrators. |
| Mitigation delivered | 22 July 2020 at 2:00 PM EDT |
| Preventive actions | Integrations from affected DSMs that rely on codegen JAR files must be updated to replace codegen functionality. Any change to an existing framework RPM, such as DSM Common or Protocol Common initiates an automatic architecture review. Test policies and automation from quality teams must be updated per architecture review to ensure that framework RPMs test against all possible combinations of log sources that might reference the codegen JAR file now or in the future. |
[{"Type":"MASTER","Line of Business":{"code":"LOB24","label":"Security Software"},"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SSBQAC","label":"IBM Security QRadar SIEM"},"ARM Category":[{"code":"a8m0z000000cwtDAAQ","label":"Auto Update"}],"Platform":[{"code":"PF016","label":"Linux"}],"Version":"7.4.0;7.4.1;7.4.2;7.4.3"},{"Type":"MASTER","Line of Business":{"code":"LOB24","label":"Security Software"},"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SSKMKU","label":"IBM QRadar on Cloud"},"ARM Category":[{"code":"a8m0z000000cwtDAAQ","label":"Auto Update"}],"Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"All Versions"}]
Was this topic helpful?
Document Information
Modified date:
02 August 2021
UID
ibm16476330