IBM Support

QRadar: Performance degradation due to reference set collisions with error "RefData_x_domain_x is experiencing heavy COLLISIONS"

Troubleshooting


Problem

Large reference sets that are not tuned and maintained, can lead to warnings related to hash collisions and may have a negative performance impact on event processing.

Symptom

Performance degradation notifications are seen under System Notifications. Look for similar warnings logged to the /var/log/qradar.log file on the console or the Event Processor:
May 11 09:27:56 ::ffff:xx.xx.xx.xx [tomcat.tomcat] [Token: TAXII Feeds@xx.xx.xx.xx (11268732) /console/restapi/api/reference_data/sets/bulk_load/XFE%2520Default%2520Feeds] com.q1labs.frameworks.cache.ChainAppendCache: [WARN] [NOT:0000004000][xx.xx.xx.xx/- -] [-/- -]RefData_57_domain_2147483647 is experiencing heavy COLLISIONS exceeding configured threshold (this may have negative performance impact) threshold = 5.0 average collisions = 10.0
 

Cause

This situation is likely to occur when a reference set does not have an appropriate Time To Live (TTL) option set and that causes the reference set to have a large number of elements.

Diagnosing The Problem

Please note the warning message:
May 11 09:27:56 ::ffff:xx.xx.xx.xx [tomcat.tomcat] [Token: TAXII Feeds@xx.xx.xx.xx (11268732) /console/restapi/api/reference_data/sets/bulk_load/XFE%2520Default%2520Feeds] com.q1labs.frameworks.cache.ChainAppendCache: [WARN] [NOT:0000004000][xx.xx.xx.xx/- -] [-/- -]RefData_57_domain_2147483647 is experiencing heavy COLLISIONS exceeding configured threshold (this may have negative performance impact) threshold = 5.0 average collisions = 10.0
 
The RefData_57_domain_2147483647 component points to a reference set with the ID 57. The name of the affected reference set can be found by using these steps. If the warning messages occur with different reference data IDs, use that ID number to locate the name of the affected reference set.
  1. Use SSH to log in to the console as root user.
  2. If the issue is not on the Console, use SSH to log in to the EP where the warning message is displayed.
  3. Run the command to find the name of the reference set and the current count of elements in it:
    psql -U qradar -c 'select id, name, current_count from reference_data where id=57;'

Resolving The Problem

  1. Open the GUI
  2. Navigate to Admin > Reference Set Management
  3. Select the reference set(s) found in the 'Diagnosing The Problem section
  4. Click Edit
  5. Set a Time To Live (TTL) value for the elements of the reference set such that it is a balance between the business purpose and the size of the reference sets. Refer to the example on editing reference data.
image 6308
Note:  Having a large number of permanent elements in a reference set is not a best practice. When a TTL value is set, it will usually take time for the older elements to be cleared from the reference set. If the performance degradation notifications and the collision warnings keep occurring even after all large reference sets have been cleared of older elements, please contact IBM QRadar support.
 

Document Location

Worldwide

[{"Line of Business":{"code":"LOB24","label":"Security Software"},"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SSBQAC","label":"IBM Security QRadar SIEM"},"ARM Category":[{"code":"a8m0z000000cwtiAAA","label":"Performance"}],"ARM Case Number":"","Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"All Version(s)"}]

Product Synonym

QRADAR, SIEM

Document Information

Modified date:
01 October 2020

UID

ibm16335163