QRadar: Understanding Traffic Analysis and Log Source Auto Detection

Traffic Analysis, also known as Auto Detection, allows QRadar to detect and create new Log Sources based on incoming event data. Traffic analysis is supported on the following protocol types:

• Syslog TCP
• Syslog UDP
• SNMP

Events received by QRadar are submitted for Auto Detection when device addresses do not yet have a matching configuration. The Traffic Analysis component performs this detection. Each event is tested against suitable DSMs to see whether it can be recognized as an event for that device type. For each device type and ip combination, statistics are kept of successful versus unsuccessful recognition for each unknown event coming through the system. After a number of events are successfully identified against a specific device type, the system creates the log source. Within a few seconds of creation, events are correctly routed through QRadar to the newly created device.

QRadar supports hundreds of log source types out of the box, with more than 150 DSMs that support automatic log source creation based on traffic analysis in QRadar. The DSM Configuration Guide includes a reference table that outlines which appliances support auto-discovery (Traffic Analysis) to create log sources from Syslog or SNMP events. If a Device or System is not discovered by Traffic Analysis, it is likely that manual log source creation is required followed by a Deploy of the thus created log source.

Appliances that run traffic analysis locally:

• 15xx - QRadar Event Collector
• 16xx - QRadar Event Processor
• 18xx - QRadar Combination Event/Flow Processor
• 21xx - QRadar Log Manager Consoles
• 31xx - QRadar Consoles
• Disconnected Log Collector (DLC overview)

For devices communicating by using other protocols, such as JDBC, Log File, Checkpoint (OPSEC/LEA), MSRPC, WMI, etc, administrators must create a log source manually before they receive event data.

When an administrator creates a log source, they must take care when they fill out the Log Source Identifier field. This field in the Log Source configuration is intended to match whatever address is in the Syslog header of the data that is received from the relevant device. If there is no Syslog header, then the IP address of the remote device, database, or log file repository must be used.

This section lists the order in which traffic analysis operates. If an administrator ran tail on /var/log/qradar.log and grep for TrafficAnalysisFilter, they would see QRadar writing events as traffic analysis works on identifying the event source or the final error message when an event source cannot be identified.

• Common log messages new log source and adding it to the traffic analysis engine

  [ecs-ec][[type=com.eventgnosis.system.ThreadedEventProcessor][parent=lab.q1labs.lab:ecs-ec/EC/TrafficAnalysis1/TrafficAnalysis]] com.q1labs.semsources.filters.trafficanalysis.TrafficAnalysisFilter: [INFO] [NOT:0000006000][x.x.x.x/- -] [-/- -]Now collecting traffic analysis statistics for address paloalto.paseries.test.com

• Attempting to create new log source

  [ecs-ec][[type=com.eventgnosis.system.ThreadedEventProcessor][parent=lab.q1labs.lab:ecs-ec/EC/TrafficAnalysis1/TrafficAnalysis]] com.q1labs.semsources.filters.trafficanalysis.TrafficAnalysisFilter: [INFO] [NOT:0000006000][x.x.x.x/- -] [-/- -]Attempting to create a new sensor device: Log Source Type = Contivityv2, Address = paloalto.paseries.test.com.

• Pausing TA on that log source identifier (Syslog host header)

  [ecs-ec][[type=com.eventgnosis.system.ThreadedEventProcessor][parent=lab.q1labs.lab:ecs-ec/EC/TrafficAnalysis1/TrafficAnalysis]] com.q1labs.semsources.filters.trafficanalysis.TrafficAnalysisFilter: [INFO] [NOT:0000006000][x.x.x.x/- -] [-/- -]Pausing traffic analysis on address paloalto.paseries.test.com while waiting for another device to be created.

• Log source failing Auto Detection

  [ecs-ec][[type=com.eventgnosis.system.ThreadedEventProcessor][parent=lab.q1labs.lab:ecs-ec/EC/TrafficAnalysis1/TrafficAnalysis]] com.q1labs.semsources.filters.trafficanalysis.TrafficAnalysisFilter: [WARN] [NOT:0070014101][x.x.x.x/- -] [-/- -]Unable to determine associated log source for IP address <x.x.x.x>. Unable to automatically detect the associated log source for IP address.

QRadar has a method for parsing event data from unsupported devices. Any device or security appliance that is not listed in the DSM Configuration Guide is considered "unsupported". Unsupported means that an existing DSM or protocol does not exist to collect and parse the events from that security device or from a specific version of an appliance.

Administrators who want to add "unsupported" devices to QRadar can create a custom log source in the DSM Editor to understand event data from security devices without a DSM provided by IBM. After you create your DSM, administrators must map the events to corresponding QRadar Identifiers (QIDs). Customers can also create their own custom QIDs to reference against events for their Universal DSM.