IBM Support

Customizing and tuning the QRadar flow configuration options

How To


Summary

The following documentation describes the flow configuration parameters in the nva.conf file. You can use these parameters to tune flow processing in QRadar.

Steps

Follow these steps to make changes to the flow configuration parameters in QRadar.
Before you begin
It is important to follow these guidelines when you make changes to the flow configuration parameters:
  • Make small, incremental changes to the parameter values.
  • Do not change many parameters at the same time. After you change a parameter value, monitor the system for awhile before you make more changes.
  • Keep track of the changes that you made and the order that you made them.
Procedure
  1. Use SSH to log in to your QRadar Console appliance as a root user.
    IMPORTANT: An administrator must back up your existing nva.conf before you attempt to make any changes. The administrator can create a folder for saved files by using the mkdir command to create /store/IBM or /store/ibmsupport for temporary files before you apply a configuration change.
  2. To back up your nva.conf file, type the following command:  cp /opt/qradar/conf/nva.conf /store/IBM/nva.conf
  3. Edit the following file:  /store/configservices/staging/globalconfig/nva.conf
  4. Update the parameters. If the parameter does not exist, add it.
  5. Save the file and then exit the SSH session by typing exit.
  6. Log in to the QRadar Console.
  7. On the Admin page, click Advanced > Deploy Full Configuration.
Configuration parameters
The following tables show the flow configuration parameters that are available in each release. The releases are cumulative so the parameters apply to each subsequent release. 
Flow configuration parameters
Parameter Description Default
7.4.0 NORMALISE_OVERFLOWED_UPTIMES
Specifies whether packet times should be adjusted when Netflow V9 sends records with overflowed system uptime values.
Options:
YES = Correct the timestamps.
NO = Do not modify the timestamps.
YES
7.4.1 QFLOW_ACCEPT_NON_ZERO_PADDING
Specifies whether to keep or discard IPFIX records with nonzero padding.
The IPFIX standard advises that the value of the padding be set to 0 for all IPFIX messages. Setting a nonzero value to padding implies that data is corrupted, or an exporter does not comply with the standards, and will be discarded by default.
Options:
YES = Discard IPFIX records with nonzero padding.
NO = Keep IPFIX records with nonzero padding.
NO
7.4.1 QFLOW_DEBUG_MESSAGE_PARSE_FAILURE
Specifies whether to generate a debug message when malformed IPFIX messages are found and discarded.
Options:
YES = Generate a debug message.
NO = Do not generate a debug message.
NO
7.4.1 QFLOW_DROP_DEBUG_SANITY_CHECK_FLOWS
Specifies the behavior when QRadar finds malformed IPFIX flow reports, or flow reports that contain unusual or unexpected data. 
Options:
YES = Drop the flow records that contain unusual or unexpected data.
NO = Keep all flow records.
To determine which fields to include in the check for malformed or unusual flow reports, use the configuration parameters that begin with QFLOW_SANITY_CHECK.
NO
7.4.1 QFLOW_SANITY_CHECK_R2R_IPS_ENABLED
Specifies whether to keep or discard IPFIX records that are tagged as Remote to Remote communication.
Options:
YES = Discard records that have Remote to Remote flow direction. 
NO = Do not check the flow direction field and keep all flow records.
To use this parameter, you must also set QFLOW_DROP_DEBUG_SANITY_CHECK_FLOWS to YES.
NO
7.4.1 QFLOW_SANITY_CHECK_TIMESTAMPS_FUTURE_ENABLED
Enables checks on IPFIX flow timestamps to look for flows that have a future timestamp that falls after a specified time. Depending on the value of this parameter, flows that match the criteria are either kept or discarded.
The future comparison time is specified in the QFLOW_SANITY_CHECK_TIMESTAMPS_FUTURE_DELTA parameter.
Options:
YES = Discard records that have timestamps that exceed the future comparison time.
NO = Do not check the timestamps and keep all flow records.
To use this parameter, you must also set QFLOW_DROP_DEBUG_SANITY_CHECK_FLOWS to YES.
NO
7.4.1 QFLOW_SANITY_CHECK_TIMESTAMPS_FUTURE_DELTA
Specifies a period of time into the future (measured in seconds), after which a flow timestamp is considered invalid.
172800
7.4.1 QFLOW_SANITY_CHECK_TIMESTAMPS_PAST_ENABLED
Enables checks on IPFIX flow timestamps to look for flows that have a past timestamp that falls after a specified time. Depending on the value of this parameter, flows that match the criteria are either kept or discarded.
The past comparison time is specified in the QFLOW_SANITY_CHECK_TIMESTAMPS_PAST_DELTA parameter.
Options:
YES = Discard records with timestamps that exceed the past comparison time.
NO = Do not check the timestamps and keep all flow records.
To use this parameter, you must also set QFLOW_DROP_DEBUG_SANITY_CHECK_FLOWS to YES.
NO
7.4.1 QFLOW_SANITY_CHECK_TIMESTAMPS_PAST_DELTA
Specifies a period of time in the past (measured in seconds), beyond which a flow timestamp is considered invalid.
172800
7.4.1 QFLOW_SANITY_CHECK_MAX_PACKETS_ENABLED
Enables checks on the flow packet field to look for flows that exceed the maximum packet size of a 32 bit unsigned integer (4,294,967,295). Depending on the value of this parameter, packet capture data that exceeds the maximum size is kept or discarded.
Options:
YES = Discard records with packet counts that exceed the maximum value.
NO = Do not check the packet count and keep all flow records.
To use this parameter, you must also set QFLOW_DROP_DEBUG_SANITY_CHECK_FLOWS to YES.
NO
7.4.1 QFLOW_SANITY_CHECK_MAX_BYTES_ENABLED
Enables checks on the byte fields to look for flows that exceed the maximum byte size of a 32 bit unsigned integer (4,294,967,295). Depending on the value of this parameter, packet capture data that exceeds the maximum size is kept or discarded.
Options:
YES = Discard records with byte counts that exceed the maximum value.
NO = Do not check the byte count and keep all flow records.
To use this parameter, you must also set QFLOW_DROP_DEBUG_SANITY_CHECK_FLOWS to YES.
NO
7.4.1 QFLOW_SANITY_CHECK_UNKNOWN_PROTOCOL_ENABLED
Enables checks on the flow protocol field to look for flows that have unknown protocols. Depending on the value of this parameter, flows that have unknown protocols are either kept or discarded.
Options:
YES = Discard flow records that have an unknown protocol.
NO = Do not check the protocol and keep all flow records. 
To use this parameter, you must also set QFLOW_DROP_DEBUG_SANITY_CHECK_FLOWS to YES.
NO
7.4.1 QNI_CONTENT_BYPASSING_SUPERFLOW
Specifies whether QRadar Network Insights content flows are included in superflow checks.
Options:
YES = Exclude QRadar Network Insights content flows.
NO = Include QRadar Network Insights content flows.
YES
7.4.1 SV_IPFIX_METADATA_ENCODING_LENGTH
Specifies the maximum length of IPFIX encoded data that is allowed in a QFlow payload, and the maximum allowable length of an IPFIX TLV data element.
TLV elements in flows that are longer than the specified length are truncated at the defined number of bytes.
In 7.4.1, the default value for this option changed from 1024 to 2048, and applies only to fields that add data up to that length.
2048
7.4.1 TIMESTAMP_OVERFLOW_THRESHOLD_PERCENT1
Specifies the threshold, measured as a percentage, for determining whether a timestamp has overflowed.
If the system uptime value of the incoming NetFlow v1/5/7/9 and IPFIX records is less than the first or last switched packet times by more than the specified percentage, the timestamps are corrected.
In QRadar versions before 7.4.1, this parameter was labeled UPTIME_OVERFLOW_THRESHOLD_MSEC.
10
7.4.1 USE_EXPORTER_SOURCE_PORT_IDENTIFIER
Specifies whether the source port is used to uniquely identify the origin of incoming IPFIX data records.
Options:
YES = Use the source port as part of the identifier.
NO = Do not use the source port.
YES
7.5.0
Update Pack 1
QFLOW_VENDOR_FLOW_ID_IN_FLOW_ID_HASHKEY
Specifies whether the vendor flow ID is included in the unique identifier for flow sessions.
Options:
YES = Include the vendor flow ID.
NO = Exclude the vendor flow ID. 
Set this parameter to NO only when flows from IBM QRadar Network Insights are to be deduplicated or asymmetrically recombined.
YES
7.5.0
Update Pack 2
QFLOW_OVERRIDE_BIFLOW_DIRECTION
Specifies whether the flow direction for bidirectional flows is set by QRadar or the flow exporter.
Options:
YES = QRadar determines the flow direction.
NO = The flow direction is set by the flow exporter. 
 
If the direction appears incorrectly for bidirectional flows, change this setting to YES to use QRadar's flow direction algorithms. 
NO
Notes
  1. IANA defines a number of different IPFIX elements for expressing the timestamp of a flow. Some of these elements are expressed as a 32-bit number, which are prone to overflowing after a period of time relative to the timescale of the element. For example, the flowStartDeltaMicroseconds element uses microseconds (μsec) and the flowStartSysUpTime element uses milliseconds (msec). Changing the configuration option from an absolute time threshold to a percentage of the potential overflow windows allows a consistent threshold across multiple timestamp methods of varying timescales. The default value of 10 percent represents a threshold of roughly 5 days for millisecond timescales and 8 minutes of microsecond timescales.
Multi-threaded processing parameters for external flow sources
The following parameters can be used for tuning multi-threaded processing for external flow sources.
Version Parameter Description Default value
7.5.0 QFLOW_EXTERNAL_TRAFFIC_MT_ON
Specifies whether multi-threaded processing is used for external flow sources.
Options:
YES = Use multi-threaded processing.
NO = Use single-threaded processing.
YES
7.5.0 QFLOW_EXTERNAL_FLOW_INPUT_BUFFER_LIMIT
Specifies the number of packets from external flow sources that are held in the buffer before traffic is dropped.
Options:
0 to 200000
100000
7.5.0 QFLOW_NUM_HASHMAPS
Defines the number of aggregation HashMaps to use for processing input data.
By default, the value of this parameter is automatically set based on hardware capacity. To override the default value, you must also set the QFLOW_OVERRIDE_EXTERNAL_TRAFFIC_MT_SETTINGS parameter to YES
The value must be equal to or greater than QFLOW_NUM_THREADS_EXTERNAL.
Calculated at startup.
7.5.0 QFLOW_NUM_THREADS_EXTERNAL
Defines the number of threads that are used for multi-threaded processing of external flow sources.
By default, the value of this parameter is automatically set based on hardware capacity. To override the default value, you must set the QFLOW_OVERRIDE_EXTERNAL_TRAFFIC_MT_SETTINGS parameter to YES.
Do not set the number of threads to be higher than the number of threads that are available on the system.
Calculated at startup.
7.5.0 QFLOW_OVERRIDE_EXTERNAL_TRAFFIC_MT_SETTINGS
Used to override the default settings for the QFLOW_NUM_THREADS_EXTERNAL parameter. 
Options:
YES = Override the default settings.
NO = Use the default settings.
NO
7.5.0
Update Pack 1
QFLOW_NUM_THREADS_REPORTER
Specifies the number of threads that are used to analyze and send flows.
By default, the value of this parameter is automatically set based on hardware capacity.
Options:
1 to 32
Calculated at startup.
7.5.0
Update Pack 1
QFLOW_NUM_CONTENT_FLOW_GC_WORKERS
Specifies the number of garbage collection threads that are used to deallocate memory for the content flow records.
By default, the value of this parameter is automatically set based on hardware capacity.
Options:
1 or 2
Calculated at startup.
7.5.0
Update Pack 1
QFLOW_NUM_DATA_FLOW_GC_WORKERS
Specifies the number of garbage collection threads that are used to deallocate memory for the data flow records. 
By default, the value of this parameter is automatically set based on hardware capacity.
Options:
1 or 2
Calculated at startup.
7.5.0
Update Pack 1
QFLOW_OVERRIDE_GARBAGE_COLLECTOR_MT_SETTINGS
Used to override the values that are used for thread assignment for garbage collector threads.
Options:
YES = Override the default settings.
NO = Use the default settings.
NO
7.5.0
Update Pack 1
QFLOW_OVERRIDE_REPORTER_MT_SETTINGS
Used to override the default settings for the QFLOW_NUM_THREADS_REPORTER parameter. 
Options:
YES = Override the default settings.
NO = Use the default settings.
NO
Multi-threaded processing parameters for raw traffic
The following parameters can be used for tuning multi-threaded processing for raw network traffic.
Version Parameter Description Default value
7.2.8 QFLOW_RAW_TRAFFIC_MT_ON
Specifies whether multi-threaded processing is used for raw traffic.
Options:
YES = Use multi-threaded processing.
NO = Use single-threaded processing.
YES
7.2.8 QFLOW_OVERRIDE_RAW_TRAFFIC_MT_SETTINGS
Used to override the values that are used for thread assignment for raw traffic.
Options:
YES = Override the default settings.
NO = Use the default settings.
NO
7.2.8 QFLOW_NUM_THREADS_NIC
Specifies the number of threads that are configured to receive input data from network interface flow sources.
By default, the value of this parameter is automatically set based on hardware capacity.
Options:
1 to 32
Calculated at startup.
7.2.8 QFLOW_NUM_THREADS_NAPATECH
Specifies the number of threads that are configured to receive input data from Napatech flow sources.
There must be sufficient Napatech host buffers configured for the specified number of threads.
Options:
1, 2, 4, 8 or 10
Calculated at startup.
Sequence number verification parameters
The following parameters can be used for tuning sequence number verification for flow records.
Version Parameter Description Default value
7.5.0
Update Pack 1
SV_VERIFY_SEQUENCE_NUMBERS
Specifies whether to use sequence number verification to detect when messages are dropped.
Options:
YES = Use sequence number verification.
NO = Turn sequence number verification off.
YES
7.5.0
Update Pack 1
QFLOW_SEQUENCE_NUMBER_GAP_REPORT_LIMIT
Used to specify the maximum number of gaps that are reported for a single exporter, per interval, before the gaps are truncated.
50

Document Location

Worldwide

[{"Type":"MASTER","Line of Business":{"code":"LOB24","label":"Security Software"},"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SSBQAC","label":"IBM Security QRadar SIEM"},"ARM Category":[{"code":"a8m0z000000cwsuAAA","label":"Flow Source"}],"ARM Case Number":"","Platform":[{"code":"PF016","label":"Linux"}],"Version":"7.4.1;7.4.2;7.4.3;7.5.0"}]

Document Information

Modified date:
31 May 2022

UID

ibm15691056