Flow improvements

The flow improvements in IBM® QRadar® 7.5.0 introduce greater visibility into flow traffic and some performance and stability improvements.

New Flow Sources API

New in 7.5.0 Update Package 7

Use the new Flow Sources API to retrieve information about the flow sources in your environment. Using this API, you can view information such as the configuration parameters that are configured for the flow source and a list of QRadar Network Insights hosts that target it. The API is read-only and you cannot use it to change the flow source information.

New information Learn more about flow sources....

Support for ICMPv6 messages

New in 7.5.0 Update Package 7

QRadar now includes support for ICMPv6 messages from NetFlow v9 and IPFIX records and expanded support for ICMPv4. The flow data now includes the following IANA IPFIX fields:
  • icmpTypeCodeIPv6 (Element ID 139)
  • icmpTypeIPv6 (Element ID 178)
  • icmpCodeIPv6 (Element ID 179)
  • icmpTypeIPv4 (Element ID 176)
  • icmpCodeIPv4 (Element ID 177)

With this enhancement, the Application and ICMP Type/Code fields show the proper ICMPv6 descriptions in the Protocol field on the Network Activity tab. In earlier versions, the ICMPv6 data was collected, but the application was shown as Other on the Network Activity tab.

If the flow record uses an ICMPv6 type and code that is unassigned by IANA, the application appears as Other.

For flow records that have IPv6 addresses but use the icmpTypeCodeIPv4 (IANA Element ID 32) field, QRadar interprets this field as icmpTypeCodeIPv6 (IANA Element ID 139). Using Element ID 139 ensures that the application ID, and the type and code, is set correctly for ICMPv6 traffic that is observed by IBM QRadar Network Insights.

Improved flow application identification

New in 7.5.0 Update Package 7

The list of common destination ports that are recognized by QRadar is expanded, making it easier to accurately identify applications when you cannot analyze the payload.

Also in this release, the common destination port configuration is updated to better reflect information in the official IANA registry.

New Common Destination Port Application ID field

New in 7.5.0 Update Package 7

With the growing trend of encrypting all network traffic, it can be a challenge to identify the type of underlying network traffic. This is difficult for applications that run on TLS encryption, such as DNS, LDAP, and RDP.

The new Common Destination Port Application ID field uses the destination port to determine the application ID. The new field is supplemental to the main application ID. For example, port 53 is a common destination port for DNS traffic. Traffic that comes in on port 53 and has a main application ID of SSL/TLS can reasonably be determined as DNS over TLS without requiring the analyst to do a manual lookup on the destination port.

Flow source domain now included in flow aggregation

New in 7.5.0 Update Package 7

The flow source domain ID is now included in the normalization key so that only flows from the same domain are aggregated. This new feature improves flow visibility in environments that have overlapping IP addresses, such as when a single flow processor monitors multiple domains.

By default, flow sources are part of the default domain. If you do not assign your flow sources to domains, no change is required.

New information Learn more about flow aggregation...

New flow direction algorithms for multicast or broadcast IP addresses

New in 7.5.0 Update Package 4
QRadar now includes two new flow direction algorithms to handle multicast or broadcast IP addresses.
  • Multicast / Broadcast Destination (reversed)
  • Multicast / Broadcast Destination (unaltered)

IP addresses 224.0.0.0 to 239.255.255.255 are multicast addresses and QRadar uses the entire range to resolve the flow direction. If a flow includes one of the reserved multicast IP addresses, or the well-known broadcast address of 255.255.255.255, QRadar automatically uses that IP address as the destination when determining the flow direction.

You can see the information in the Flow Direction Algorithm field on the Flow Information window.

New information Learn more about flow direction...

Improved visibility for NetFlow V9 flow records

New in 7.5.0 Update Package 4

IBM QRadar now supports the firewallEvent field (IANA element 233) in NetFlow V9 data exports. In QRadar, the Firewall Event field appears in the Flow Data section of the Flow Details window. You can use the firewallEvent field in filters, searches, and rules.

New information Learn more about NetFlow flow sources...

Support for IPFIX bidirectional flows

New in 7.5.0 Update Package 2

IBM QRadar now supports bidirectional flows for the IP Flow Information Export (IPFIX) protocol.

If your IPFIX exporter supports bidirectional flows, you might see performance improvements due to increased throughput from the exporter.

Multi-threaded processing for external flow sources

Changed in 7.5.0 Update Package 1

As part of ongoing improvements to the flow pipeline, the QFlow flow processing service now supports multi-threaded processing for external flow sources, such as IPFIX, NetFlow V9, and IBM QRadar Network Insights flow sources.

QRadar 7.5.0 introduced multi-threaded processing in the receiving, parsing, and normalization phases when processing external flow sources.

Building on those improvements, QRadar 7.5.0 Update Package 1 introduces multi-threaded processing for the analysis, sending, and garbage collection of flows within the QFlow flow processing service.

Now that the entire QFlow flow processing service uses multi-threaded processing. This change improves the performance of the QFlow process and allows QRadar to process more flows.

Multi-threaded processing is turned on by default, and the number of threads is automatically determined based on the capabilities of the appliance.

Sequence number verification

New in 7.5.0 Update Package 1

Now that all stages of the QFlow process uses multi-threaded processing, QRadar can use sequence number verification to detect when messages are dropped. Dropped messages might indicate that something is wrong in your network, such as a faulty flow exporter, a lossy network, or packet injection into the network by an attacker.

In QRadar 7.5.0 Update Package 1, missing sequence numbers are reported only once per minute to ensure that packets have time to fill in gaps in the sequence ranges. Missed sequence numbers are reported in the /var/log/qradar.log file.

Support for Network Address Translation fields

QRadar can now receive network address translation (NAT) information from IPFIX and NetFlow V9 flow records.

The following NAT fields are supported in QRadar 7.5.0:
  • postNATSourceIPv4Address (IANA Element ID 225)
  • postNATDestinationIPv4Address (IANA Element ID 226)
  • postNAPTSourceTransportPort (IANA Element ID 227)
  • postNAPTDestinationTransportPort (IANA Element ID 228)

The new fields are categorized under Flow Data on the Flow Information window. You can use them in filters, searches, and rules.

New information Learn more about the fields that are supported by QRadar flow sources...

New application determination algorithms

Now you can see more information about the application identification algorithm that is used for IBM QRadar Network Insights flows.

The QNI Inspectors (9) algorithm is removed in this release. It is replaced by the following new algorithms:
  • QNI port heuristics (11)

    This algorithm is used when QRadar Network Insights identifies the application based on port heuristics. It represents the least degree of confidence in the application determination.

  • QNI initial data (12)

    This algorithm is used when QRadar Network Insights identifies the application based on the analysis of initial data in the flow session. It represents a medium degree of confidence.

  • QNI parsers (13)

    This algorithm is used when QRadar Network Insights is confident in determining the application based on the data that is available.

You can see the information in the Application Determination Algorithm field on the Flow Information window.

New information Learn more about identifying flow applications...

Support for more fields from AWS VPC flow logs

QRadar now shows more information from Amazon Web Services (AWS) Virtual Private Cloud (VPC) Version 3 flow logs.

QRadar 7.5.0 supports the following fields:
  • VPC ID
  • Subnet ID
  • Instance ID

When an IPFIX flow record includes these fields, QRadar shows the information on the Flow Details page under the Cloud category.

New information Learn more about viewing AWS flow data...

More improvements

IBM QRadar 7.5.0 also includes the following enhancements:
  • On the Component Management window (Admin > System and License Management > Edit Host), the Alias Autodetection field is renamed to DNS lookup for Alias Autodetection.
  • The flow direction algorithm is now applied at the beginning of the flow parsing process.

    This change ensures that the destination port is determined before the payload content capture occurs so that the amount of captured payload always matches the setting in the common destination port configuration.

  • Only the relevant IPFIX fields are encoded into the payload.

    The default encoding method for some IPFIX fields changed, and they are no longer appended to the payload. Now, they are added to the flow as type-value-length (TLV) elements.

  • You cannot delete the Uncategorized category for tagged flow fields from your system.