IBM Support

QRadar: Understanding Tenant EPS and FPM limit rate

Question & Answer


Question

Why is my tenant EPS or FPM limit rate not working properly and my tenants are exceeding their limit? 

Cause

At times QRadar administrators can encounter error notifications with logs such as the following:
Jun 16 09:53:07 Event dropped while attempting to add to Tenant Event Throttle queue. 
The Tenant Event Throttle queue is full. [TenantID: 1, Tenant name: TEST, dropped in 
last interval: 8698, total dropped: 7864793]
From seeing these notifications, administrators often review their dashboards in order to verify how much data the tenants are sending and can encounter that the EPS or FPM limits are being exceeded.
You can use the following steps to determine the average EPS per domain from the last five minutes by using an AQL search:
  1. Log in to the QRadar user interface.
  2. Go to the Log Activity.
  3. Run the following search on the Advanced Search bar:
    select domainname(domainid) as 'Domain', count(*)/300 as 'EPS' from events last 5 minutes
    Output example:
    The EPS for the Default Domain is 1742.67.
    image-20230518084155-1

    Result
    The administrator is able to see the EPS for the last five minutes by domain.
The question arises, is the EPS or FPM rate limit not working correctly? However, in fact, the rate limit is working correctly, there are two default parameters that come into play:
  • TENANT_QUEUE_SIZE_MILTIPLIER
  • TENANT_QUEUE_THREAD_INTERVAL_IN_MILLISEC

Answer

These two parameters have the following default values:
  • TENANT_QUEUE_SIZE_MILTIPLIER: 1.5
  • TENANT_QUEUE_THREAD_INTERVAL_IN_MILLISEC: 400
These values mean that if you set up a limit of 100 EPS for a Tenant, it is able to handle up to 150 EPS for 400 milliseconds.
Due to these two parameters, we can get the wrong impression that the limits are not applied, however they are implemented so the Tenants can have the chance to keep up with the incoming EPS peaks and avoid dropping a large volume of events.
IMPORTANT: QRadar Support Team does not support the edition of these numbers.

[{"Type":"MASTER","Line of Business":{"code":"LOB24","label":"Security Software"},"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SSBQAC","label":"IBM Security QRadar SIEM"},"ARM Category":[{"code":"a8m0z000000cwtiAAA","label":"Performance"}],"ARM Case Number":"","Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"All Versions"}]

Document Information

Modified date:
24 May 2023

UID

ibm16986229