WinCollect: Troubleshooting missing heartbeat events from WinCollect agent

Troubleshooting

Problem

This article helps you troubleshoot a scenario when you are not seeing any heartbeat events from a specific WinCollect agent.

Symptom

Any or all the symptoms might exist:

The status of the WinCollect agent is "Unavailable".
The WinCollect DSM log source is in an error state.
Heartbeat events cannot be seen in Log Activity.

Cause

Common causes are:

The WinCollect service is not running on the Windows® host where WinCollect is installed.
The StatusServer IP address or hostname is incorrect in C:\Program Files\IBM\WinCollect\config\install_config.txt.
A connectivity issue; UDP traffic to the QRadar event collector on port 514 might be blocked.
Multiple heartbeat log sources for the WinCollect Agent exist in the QRadar PostgreSQL database.

Diagnosing The Problem

What WinCollect "heartbeat" events are

A WinCollect heartbeat is a message sent every 5 minutes from the WinCollect agent to a Status-Server. The message contains specific information, including the hostname. Sample payload:

<13>Jul 06 22:14:47 WIN-C79QC3HVT51 LEEF:1.0|IBM|WinCollect|7.3.1.28|2|src=WIN-C79QC3HVT51	os=Windows Server 2022 Datacenter (Build 20348 64-bit)	dst=x.x.x.x	sev=3	log=Code.SSLConfigServerConnection	msg=ApplicationHeartbeat

By default, heartbeat events are sent on the 514 UDP port to the defined StatusServer. This configuration is present on the Windows® hosts in C:\ProgramFiles\IBM\WinCollect\config\AgentConfig.xml file.

    <Service name="AgentCore" module="AgentCore" type="Service" classification="Static" version="7.3.1-28">
        <Environment>
            <Parameter name="HeartbeatInterval" value="300000"/>
            <Parameter name="LogMonitor.Socket.Type" value="UDP"/>
            <Parameter name="ConfigurationCheckInterval" value="300000"/>
            <Parameter name="Enabled" value="true"/>
            <Parameter name="Deleted" value="false"/>
        </Environment>
    </Service>

This configuration can also be seen from the QRadar side, Admin > Wincollect > Agents:

Resolving The Problem

Wincollect Heartbeat log source.

After installation of the WinCollect, the Heartbeat log source is created with the name WinCollect DSM - Agent_name with log source type WinCollect and protocol Syslog. If this installation is a first-time installation, you see that this log source is pending a Deploy in the Admin tab in the yellow section.
Example:
image-20240707125642-1

This log source is associated with the agent in the PostgreSQL database in the ale_client table in QRadar.
Example query:

psql -U qradar -x -c "select * from ale_client where hostname ilike '%Agent_name%';"

Example output:

image-20240707130202-2

Troubleshooting Missing heartbeat events.

Solution 1: Check that the WinCollect service is running on the Windows host where WinCollect is installed.
image-20240707173641-1

Solution 2: Check that the StatusServer IP address or the FQDN is correct in the C:\ProgramFiles\IBM\WinCollect\config\install_config.xml file on Windows host.
Example:

ApplicationIdentifier=Agent-hostname
LocalIP=
OriginatingComputer=
ConfigurationServer=10.10.10.11
ConfigurationServerPort=8413
ConfigurationServerMinSSLProtocol=TLSv1
ConfigurationServerMaxSSLProtocol=TLSv1.2
StatusServer=10.10.10.10
ApplicationToken=U59SSDt/u1kPE.........
BuildNumber=43

Solution 3: To verify the event in a Tcpdump packet capture, run the following command on the StatusServer command line interface:

tcpdump -nnAs0 -i any host <Wincollect IP> and port 514 | grep -i heartbeat

Solution 4: If events are not visible in the Tcpdump output, and there is a firewall between the WinCollect host and StatusServer, check with your local network team whether UDP traffic on port 514 is allowed or not from the WinCollect host to the StatusServer.

Solution 5: If events are visible in the Tcpdump output, but the agent status is "Unavailable" and the WinCollect DSM-log source is in an error state, verify whether there are multiple log sources present or not in the PSQL database and raise an IBM QRadar Support case for further investigation. Include Getlogs from the Console and screen captures to illustrate the issue.
Example query:

psql -U qradar -x -c "select * from sensordevice where hostname like '%Agent_name%';"

Document Location

Worldwide

[{"Type":"MASTER","Line of Business":{"code":"LOB24","label":"Security Software"},"Business Unit":{"code":"BU048","label":"IBM Software"},"Product":{"code":"SSBQAC","label":"IBM Security QRadar SIEM"},"ARM Category":[{"code":"a8m0z000000cwtwAAA","label":"WinCollect"}],"ARM Case Number":"","Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"All Versions"}]

Tips