IBM Support

QRadar: How to verify data sent from an Event Collector is processed

Troubleshooting


Problem

Verifying that data is being sent from an Event Collector is helpful in the following use-cases:
  • To ensure that the event data from the specific Event Collector is processed continuously  
  • To identify any potential network connectivity issues between Event Collector and the Event Processor (or Console)
  • To find any potential gaps within event data flow
  • To detect any system malfunction on the Event Collector side (for instance system or hardware issues)

Symptom

If you experience these symptoms, the Event Collector might be failing to send data:
  • The QRadar Log Source Management app reports that some of the Log Sources are the error state
  • Search results depict event gaps

  • Event Collector intermittently display status Unknown

Cause

Any of the following can cause the Event Collector to fail to send data:
  • Intermittent network connectivity issues
  • Low bandwidth 
  • Persistent over-license condition
  • Services instability on an Event Collector
  • Varies hardware (or hypervisor) issues that affect an Event Collector system
  • Misconfigured intermediary network devices (Firewalls, IPS/IDS) on the link between an Event Collector and Event Processor (Console)

Diagnosing The Problem

Before you begin
To conduct the verification process, you need an IP address of the Event Collector network management interface.  It can be obtained from the GUI by going to Admin > System and License Management or from the CLI by running the following command on the console: 
grep -i '${Event Collector hostname}' /etc/hosts
CLI
  1. Use SSH to log in to the Console as root user.
  2. To obtain the Managed Host ID of the target Event Collector edit and run the following command on the console: 
    psql -U qradar -c "SELECT id,ip FROM managedhost WHERE ip='$<YOUR EVENT COLLECTOR IP>';"
    Note down the Managed Host ID.  In this example, the id is 106.
    Step 1 | Obtain the Managed Host ID
  3. Obtain the Event Collector ID based on the managed host ID from the previous step by editing and running the following command: 
    psql -U qradar -c "SELECT id,name FROM deployed_component WHERE managed_host_id=$<YOUR MANAGED HOST ID> AND NAME ILIKE 'e%' AND CHAR_LENGTH(name) < 20;"
    In this example, the id is 118.
    Step 3 | obtaining the Event Collector ID

    Result
    If the search returned no events, then proceed to Resolving the Problem.
GUI
  1. To create a customized search, go to Log Activity > Search > New Search.Step 4 | New Search
  2. Define the Search Mode, Time Range, and Search Parameters.Step 4 | Customized Search
    The second search parameter 'Destination IP is not 127.0.0.1' limits search results only to the Log Sources event data:
    Step 4a | Search parameters
  3. Run the customized search.
  4. Analyze the search results:Step 5 | Analyze the search results

    Result
    Events appear shortly after the customized search was run (same as on the animation). You can update search results with any other time frame (go to 'View' dropdown menu) to identify any potential event data gaps.
    Important: If the search returned no events, then proceed to Resolving the Problem.

Resolving The Problem

  1. From the Console, open an SSH session to the target Event Collector.
    Note: If you are unsuccessful with this step, start further troubleshooting with the following document: QRadar: Verifying SSH connectivity to the target Managed Host. Continue with the next step when the issue is resolved. 
  2. Check whether the services are running on the target Event Collector: 
    /opt/qradar/upgrade/util/setup/upgrades/wait_for_start.sh
    This example shows the services are running:
    Step 8 | Check EC services

    1. If the services are not running, restart the core system services on the target Event collector:

      systemctl stop hostcontext
systemctl restart hostservices
systemctl start hostcontext

      After a few minutes, check again if the services are running.

      /opt/qradar/upgrade/util/setup/upgrades/wait_for_start.sh

      If the services are still not running after the restart proceed to collect the log files and open a case with QRadar Support. Continue with the next step when the issue is resolved. 

  3. Verify that the target Event Collector receives event data from the configured Log Sources. 
    1. Obtain the Event Collector network interface name by using the following command: 
      cat /etc/management_interface
      In this example, the interface is ens192:
      Step 9a | an EC network interface
    2. Run the tcpdump command to check whether event data is reaching the target Event Collector. 
      tcpdump -nnAs0 -i ${EC network interface} port 514 -c 10
      The tcpdump stops when ten network packets with the Log Source event data are captured:
      Step 9b | TCP Dump
       
    Result
    If the command returns packets, then the Event Collector is receiving event data. 
    If no output is present, then the next troubleshooting steps are to check the Log Source configuration and verify whether the network traffic from Log sources is not blocked by the firewalls.

Document Location

Worldwide

[{"Type":"MASTER","Line of Business":{"code":"LOB24","label":"Security Software"},"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SSV4BL","label":"IBM QRadar"},"ARM Category":[{"code":"a8m0z000000cwtEAAQ","label":"Log Activity"}],"ARM Case Number":"","Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"All Versions"}]

Document Information

Modified date:
03 April 2023

UID

ibm16848271

Page Feedback