IBM Support

QRadar: How to calculate the storage used by events and flows per day on a QRadar host

How To


Summary

This article serves as a guide to calculate how much storage is being used daily for both events and flows in the present month by using the command-line interface.

Environment

 The commands in this article can be executed on QRadar hosts with local storage, which includes:
  • All-in-One consoles.
  • Event and Flow Processors.
  • Flow Processors.
  • Event Processors.
  • Data Nodes.
Before you begin
  • Some of these commands can be long, make sure to copy the entire line.
  • Depending on the amount of data stored on the host, the output might take longer to be displayed.

Steps

The commands to calculate the storage use per month varies depending on the tenant configuration. Follow the next steps per environment type to get the needed information.

Single-tenant environments:

  1. Log in to QRadar by using the CLI.
  2. The command needs to be executed on the device that is storing the events, so if the events are on a device different than the console, ssh to that device to execute the command.
  3. Execute the next command, these commands generate a text file with the needed information.
    For events: 
    for i in $(eval echo {1..$(date +%-d)}); do echo $(date +%Y)"/"$(date +%-m)"/"$i && du -shc /store/ariel/events/records/$(date +%Y)/$(date +%-m)/$i /store/ariel/events/payloads/$(date +%Y)/$(date +%-m)/$i 2>/dev/null ; echo  ;done > dailyStorageUsedByEvents-$(date +%B)$(date +%Y).txt
    For Flows: 
    for i in $(eval echo {1..$(date +%-d)}); do echo $(date +%Y)"/"$(date +%-m)"/"$i && du -shc /store/ariel/flows/records/$(date +%Y)/$(date +%-m)/$i 2>/dev/null ; echo  ;done > dailyStorageUsedByFlows-$(date +%B)$(date +%Y).txt
  4. Result:
    The commands generate a .txt file, for example dailyStorageUsedByEvents-July2022.txt with the amount of stored data per day: 
    image-20220719140413-1

Multi-tenant environments:

When multi-tenancy is enabled and retention buckets are configured, each tenant has its own directory in the file system, they are located in the next path:

/store/ariel/events/records/aux/<tenantID#>/Year/Month/Day 

These directories contain the information for those specific tenants.

The default path /store/ariel/events/records|payloads/ still exists for any nontenant tagged data. 

Follow the next steps to get the information for multi-tenant environments:

  1. In order to check the storage used by a specific tenant, we need to identify the ID of that tenant. The following command can be used to get the name associated with the tenant ID: 
    psql -U qradar -c "SELECT id, name FROM tenant WHERE deleted='f';"
  2. Once the ID of the tenant is identified, the following commands can be executed.
    For events: 
    Make sure to replace <Tenant_ID> with the corresponding value. It has to be replaced three times in this command.
    for i in $(eval echo {1..$(date +%-d)}); do echo $(date +%Y)"/"$(date +%-m)"/"$i && du -shc /store/ariel/events/records/aux/<Tenant_ID>/$(date +%Y)/$(date +%-m)/$i /store/ariel/events/payloads/aux/<Tenant_ID>/$(date +%Y)/$(date +%-m)/$i 2>/dev/null ; echo  ;done > dailyStorageUsedByEventsTenant<Tenant_ID>-$(date +%B)$(date +%Y).txt
    For flows:
    The <Tenant_ID> needs to be replaced two times in this command. 
    for i in $(eval echo {1..$(date +%-d)}); do echo $(date +%Y)"/"$(date +%-m)"/"$i && du -shc /store/ariel/flows/records/aux/<Tenant_ID>/$(date +%Y)/$(date +%-m)/$i 2>/dev/null ; echo  ;done > dailyStorageUsedByFlowsTenant<Tenant_ID>-$(date +%B)$(date +%Y).txt
  3. Result:
    The commands generate a .txt file, for example dailyStorageUsedByEventsTenant1July2022.txt with the amount of stored data per day: 
    image-20220719143012-2

Document Location

Worldwide

[{"Type":"MASTER","Line of Business":{"code":"LOB24","label":"Security Software"},"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SSBQAC","label":"IBM Security QRadar SIEM"},"ARM Category":[{"code":"a8m0z000000cwt8AAA","label":"Ariel"}],"ARM Case Number":"","Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"All Versions"}]

Document Information

Modified date:
20 February 2023

UID

ibm16601755

Page Feedback