QRadar: Process Monitor: Application has failed to start up

Troubleshooting

Problem

Using a Flow Collector connected to a Flow Processor, if the Flow Processor is rebuilt, the Flow Collector can no longer communicate to the Flow Processor

Symptom

Multiple Process Monitor: Application has failed to start up multiple times Notifications.

Cause

SSH host keys on the Flow Collector for the Flow Processor are old and invalid.
A Flow source is missing.

Environment

Distributed environment with Flow Collectors communicating with Flow Processors.

Diagnosing The Problem

To determine which of the two causes are contributing to the problem.

Verifying Cause One

Using SSH, log into the Console(CLI).
SSH from the Console(CLI) to the Flow Collector.
Attempt to SSH to the Flow Processor. This will fail with an error stating which key in the /root/.ssh/known_hosts file is incorrect.

Verifying Cause Two

Log into the QRadar web User Interface.
Click on the Admin tab > Flow Sources icon.
Verify there is one External Flow Source and one Internal Flow Source connection for each Target Flow Collector.
External Flow Sources options are NetFlow, IPFIX, Flow Log File, JFlow, SFlow, and Packeteer.
Internal Flow Sources options are Network Interface, and Napatech interface.

Resolving The Problem

Resolving Cause One

Log into the QRadar Console using SSH.
SSH from the Console into the Flow Processor.
Backup /root/.ssh/known_hosts by typing the following:
cp /root/.ssh/known_hosts /root/known_hosts
Remove the indicated key for the old Flow Processor in the /root/.ssh/known_hosts file.
The old key will have the IP Address of the Flow Collector.

Resolving Cause Two

Log into the QRadar web User Interface.
Click on the Admin tab > Flow Sources icon.
Click on Add.
Add the appropriate Flow Sources for your system, You need both an External Flow Source and an Internal Flow Source for each Target Flow Collector.
From the top menu banner click on Deploy Changes.

For QRadar versions prior to 7.3, alternatively you can open the Deployment Editor and remove the Flow Process.

You need to delete the Connection line and QFlow object.
You then need to remove the unassigned Target Flow Collector from Flow Sources.

Example:

Where do you find more information?

Related Information

Flow Sources

TroubleShooting System Notification Guide

[{"Product":{"code":"SSBQAC","label":"IBM Security QRadar SIEM"},"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Component":"Operating System","Platform":[{"code":"PF016","label":"Linux"}],"Version":"7.2;7.3","Edition":"","Line of Business":{"code":"LOB24","label":"Security Software"}}]

Tips