How To
Summary
The following documentation describes the flow configuration parameters in the nva.conf file. You can use these parameters to tune flow processing in QRadar.
Steps
Follow these steps to make changes to the flow configuration parameters in QRadar.
Before you begin
It is important to follow these guidelines when you make changes to the flow configuration parameters:
- Make small, incremental changes to the parameter values.
- Do not change many parameters at the same time. After you change a parameter value, monitor the system for awhile before you make more changes.
- Keep track of the changes that you made and the order that you made them.
Procedure
- Use SSH to log in to your QRadar Console appliance as a root user.
IMPORTANT: An administrator must back up your existing nva.conf before you attempt to make any changes. The administrator can create a folder for saved files by using the mkdir command to create /store/IBM or /store/ibmsupport for temporary files before you apply a configuration change. - To back up your nva.conf file, type the following command: cp /opt/qradar/conf/nva.conf /store/IBM/nva.conf
- Edit the following file: /store/configservices/staging/globalconfig/nva.conf
- Update the parameters. If the parameter does not exist, add it.
- Save the file and then exit the SSH session by typing exit.
- Log in to the QRadar Console.
- On the Admin page, click
Advanced > Deploy Full Configuration.
Configuration parameters
The following tables show the flow configuration parameters that are available in each release. The releases are cumulative so the parameters apply to each subsequent release.
- Flow configuration parameters
- Multi-threaded processing parameters for raw traffic
- Multi-threaded processing parameters for external flow sources
- Sequence number verification parameters
Flow configuration parameters
| Parameter | Description | Default | |
|---|---|---|---|
| 7.4.0 | NORMALISE_OVERFLOWED_UPTIMES |
Specifies whether packet times should be adjusted when Netflow V9 sends records with overflowed system uptime values.
Options:
YES = Correct the timestamps.
NO = Do not modify the timestamps.
|
YES |
| 7.4.1 | QFLOW_ACCEPT_NON_ZERO_PADDING |
Specifies the way that QRadar handles flows with non-zero padding.
In QRadar 7.5.0 Update Pack 1 or earlier, this parameter specifies whether to keep or discard IPFIX records with nonzero padding.
YES = Keep IPFIX records.
NO = Discard IPFIX records. (Default)
In QRadar 7.5.0 Update Pack 2 or later, this parameter specifies whether a message is logged when QRadar receives an IPFIX and NetFlow V9 flow with non-zero padding. The flows are still processed, regardless of which option you choose.
YES = Suppress the log message.
NO = Log a message. (Default)
|
Version-specific |
| 7.4.1 | QFLOW_DEBUG_MESSAGE_PARSE_FAILURE |
Specifies whether to generate a debug message when malformed IPFIX messages are found and discarded.
Options:
YES = Generate a debug message.
NO = Do not generate a debug message.
|
NO |
| 7.4.1 | QFLOW_DROP_DEBUG_SANITY_CHECK_FLOWS |
Specifies the behavior when QRadar finds malformed IPFIX flow reports, or flow reports that contain unusual or unexpected data.
Options:
YES = Drop the flow records that contain unusual or unexpected data.
NO = Keep all flow records.
To determine which fields to include in the check for malformed or unusual flow reports, use the configuration parameters that begin with QFLOW_SANITY_CHECK.
|
NO |
| 7.4.1 | QFLOW_SANITY_CHECK_R2R_IPS_ENABLED |
Specifies whether to keep or discard IPFIX records that are tagged as Remote to Remote communication.
Options:
YES = Discard records that have Remote to Remote flow direction.
NO = Do not check the flow direction field and keep all flow records.
To use this parameter, you must also set QFLOW_DROP_DEBUG_SANITY_CHECK_FLOWS to YES.
|
NO |
| 7.4.1 | QFLOW_SANITY_CHECK_TIMESTAMPS_FUTURE_ENABLED |
Enables checks on IPFIX flow timestamps to look for flows that have a future timestamp that falls after a specified time. Depending on the value of this parameter, flows that match the criteria are either kept or discarded.
The future comparison time is specified in the QFLOW_SANITY_CHECK_TIMESTAMPS_FUTURE_DELTA parameter.
Options:
YES = Discard records that have timestamps that exceed the future comparison time.
NO = Do not check the timestamps and keep all flow records.
To use this parameter, you must also set QFLOW_DROP_DEBUG_SANITY_CHECK_FLOWS to YES.
|
NO |
| 7.4.1 | QFLOW_SANITY_CHECK_TIMESTAMPS_FUTURE_DELTA |
Specifies a period of time into the future (measured in seconds), after which a flow timestamp is considered invalid.
|
172800 |
| 7.4.1 | QFLOW_SANITY_CHECK_TIMESTAMPS_PAST_ENABLED |
Enables checks on IPFIX flow timestamps to look for flows that have a past timestamp that falls after a specified time. Depending on the value of this parameter, flows that match the criteria are either kept or discarded.
The past comparison time is specified in the QFLOW_SANITY_CHECK_TIMESTAMPS_PAST_DELTA parameter.
Options:
YES = Discard records with timestamps that exceed the past comparison time.
NO = Do not check the timestamps and keep all flow records.
To use this parameter, you must also set QFLOW_DROP_DEBUG_SANITY_CHECK_FLOWS to YES.
|
NO |
| 7.4.1 | QFLOW_SANITY_CHECK_TIMESTAMPS_PAST_DELTA |
Specifies a period of time in the past (measured in seconds), beyond which a flow timestamp is considered invalid.
|
172800 |
| 7.4.1 | QFLOW_SANITY_CHECK_MAX_PACKETS_ENABLED |
Enables checks on the flow packet field to look for flows that exceed the maximum packet size of a 32 bit unsigned integer (4,294,967,295). Depending on the value of this parameter, packet capture data that exceeds the maximum size is kept or discarded.
Options:
YES = Discard records with packet counts that exceed the maximum value.
NO = Do not check the packet count and keep all flow records.
To use this parameter, you must also set QFLOW_DROP_DEBUG_SANITY_CHECK_FLOWS to YES.
|
NO |
| 7.4.1 | QFLOW_SANITY_CHECK_MAX_BYTES_ENABLED |
Enables checks on the byte fields to look for flows that exceed the maximum byte size of a 32 bit unsigned integer (4,294,967,295). Depending on the value of this parameter, packet capture data that exceeds the maximum size is kept or discarded.
Options:
YES = Discard records with byte counts that exceed the maximum value.
NO = Do not check the byte count and keep all flow records.
To use this parameter, you must also set QFLOW_DROP_DEBUG_SANITY_CHECK_FLOWS to YES.
|
NO |
| 7.4.1 | QFLOW_SANITY_CHECK_UNKNOWN_PROTOCOL_ENABLED |
Enables checks on the flow protocol field to look for flows that have unknown protocols. Depending on the value of this parameter, flows that have unknown protocols are either kept or discarded.
Options:
YES = Discard flow records that have an unknown protocol.
NO = Do not check the protocol and keep all flow records.
To use this parameter, you must also set QFLOW_DROP_DEBUG_SANITY_CHECK_FLOWS to YES.
|
NO |
| 7.4.1 | QNI_CONTENT_BYPASSING_SUPERFLOW |
Specifies whether QRadar Network Insights content flows are included in superflow checks.
Options:
YES = Exclude QRadar Network Insights content flows.
NO = Include QRadar Network Insights content flows.
|
YES |
| 7.4.1 | SV_IPFIX_METADATA_ENCODING_LENGTH |
Specifies the maximum length of IPFIX encoded data that is allowed in a QFlow payload, and the maximum allowable length of an IPFIX TLV data element.
TLV elements in flows that are longer than the specified length are truncated at the defined number of bytes.
In 7.4.1, the default value for this option changed from 1024 to 2048, and applies only to fields that add data up to that length.
|
2048 |
| 7.4.1 | TIMESTAMP_OVERFLOW_THRESHOLD_PERCENT1 |
Specifies the threshold, measured as a percentage, for determining whether a timestamp has overflowed.
If the system uptime value of the incoming NetFlow v1/5/7/9 and IPFIX records is less than the first or last switched packet times by more than the specified percentage, the timestamps are corrected.
In QRadar versions before 7.4.1, this parameter was labeled UPTIME_OVERFLOW_THRESHOLD_MSEC.
|
10 |
| 7.4.1 | USE_EXPORTER_SOURCE_PORT_IDENTIFIER |
Specifies whether the source port is used to uniquely identify the origin of incoming IPFIX data records.
Options:
YES = Use the source port as part of the identifier.
NO = Do not use the source port. |
YES |
|
7.5.0
Update Pack 1
|
QFLOW_VENDOR_FLOW_ID_IN_FLOW_ID_HASHKEY |
Specifies whether the vendor flow ID is included in the unique identifier for flow sessions.
Options:
YES = Include the vendor flow ID.
NO = Exclude the vendor flow ID.
Set this parameter to NO only when flows from IBM QRadar Network Insights are to be deduplicated or asymmetrically recombined.
|
YES |
|
7.5.0
Update Pack 2
|
QFLOW_OVERRIDE_BIFLOW_DIRECTION |
Specifies whether the flow direction for bidirectional flows is set by QRadar or the flow exporter.
Options:
YES = QRadar determines the flow direction. NO = The flow direction is set by the flow exporter. If the direction appears incorrectly for bidirectional flows, change this setting to YES to use QRadar's flow direction algorithms. |
NO |
|
7.5.0
Update Pack 6
|
SV_NETFLOW_FLOW_FORWARD_PORT_LIMIT | Sets the maximum number of available ports to be used for flow forwarding. This setting is configurable up to a maximum of 500 ports. | 200 |
|
7.5.0
Update Pack 6
|
SV_NETFLOW_FLOW_FORWARD_CLOSE_STALE_PORT_ON_EXHAUSTION |
Specifies whether the QFlow process closes the most stale port when a new exporter comes online after the maximum number of ports is reached.
Options:
YES = QFlow closes the most stale port, reassigns it to the new exporter, and forwards the flows. NO = Flows are not forwarded. After a port becomes available, QRadar begins to forward flows from that exporter. |
YES |
|
7.5.0
Update Pack 6
|
SV_NETFLOW_FLOW_FORWARD_STALE_PORT_CHECK_PERIOD | Sets the period, measured in seconds, of how often QFlow checks to determine whether any ports are stale and ready to be reassigned. The default is 300 seconds (5 minutes). | 300 |
|
7.5.0
Update Pack 6
|
SV_NETFLOW_FLOW_FORWARD_STALE_PORT_LIFETIME | Sets the period, measured in seconds, of how long a port is unused before it is considered stale. The default is 1800 seconds (30 minutes). | 1800 |
Notes
-
IANA defines a number of different IPFIX elements for expressing the timestamp of a flow. Some of these elements are expressed as a 32-bit number, which are prone to overflowing after a period of time relative to the timescale of the element. For example, the flowStartDeltaMicroseconds element uses microseconds (μsec) and the flowStartSysUpTime element uses milliseconds (msec). Changing the configuration option from an absolute time threshold to a percentage of the potential overflow windows allows a consistent threshold across multiple timestamp methods of varying timescales. The default value of 10 percent represents a threshold of roughly 5 days for millisecond timescales and 8 minutes of microsecond timescales.
Multi-threaded processing parameters for external flow sources
The following parameters can be used for tuning multi-threaded processing for external flow sources.
| Version | Parameter | Description | Default value |
|---|---|---|---|
| 7.5.0 | QFLOW_EXTERNAL_TRAFFIC_MT_ON |
Specifies whether multi-threaded processing is used for external flow sources.
Options:
YES = Use multi-threaded processing.
NO = Use single-threaded processing.
|
YES |
| 7.5.0 | QFLOW_EXTERNAL_FLOW_INPUT_BUFFER_LIMIT |
Specifies the number of packets from external flow sources that are held in the buffer before traffic is dropped.
Options:
0 to 200000
|
100000 |
| 7.5.0 | QFLOW_NUM_HASHMAPS |
Defines the number of aggregation HashMaps to use for processing input data.
By default, the value of this parameter is automatically set based on hardware capacity. To override the default value, you must also set the QFLOW_OVERRIDE_EXTERNAL_TRAFFIC_MT_SETTINGS parameter to YES.
The value must be equal to or greater than QFLOW_NUM_THREADS_EXTERNAL.
|
Calculated at startup. |
| 7.5.0 | QFLOW_NUM_THREADS_EXTERNAL |
Defines the number of threads that are used for multi-threaded processing of external flow sources.
By default, the value of this parameter is automatically set based on hardware capacity. To override the default value, you must set the QFLOW_OVERRIDE_EXTERNAL_TRAFFIC_MT_SETTINGS parameter to YES.
Do not set the number of threads to be higher than the number of threads that are available on the system.
|
Calculated at startup. |
| 7.5.0 | QFLOW_OVERRIDE_EXTERNAL_TRAFFIC_MT_SETTINGS |
Used to override the default settings for the QFLOW_NUM_THREADS_EXTERNAL parameter.
Options:
YES = Override the default settings.
NO = Use the default settings.
|
NO |
|
7.5.0
Update Pack 1
|
QFLOW_NUM_THREADS_REPORTER |
Specifies the number of threads that are used to analyze and send flows.
By default, the value of this parameter is automatically set based on hardware capacity.
Options:
1 to 32
|
Calculated at startup. |
|
7.5.0
Update Pack 1
|
QFLOW_NUM_CONTENT_FLOW_GC_WORKERS |
Specifies the number of garbage collection threads that are used to deallocate memory for the content flow records.
By default, the value of this parameter is automatically set based on hardware capacity.
Options:
1 or 2
|
Calculated at startup.
|
|
7.5.0
Update Pack 1
|
QFLOW_NUM_DATA_FLOW_GC_WORKERS |
Specifies the number of garbage collection threads that are used to deallocate memory for the data flow records.
By default, the value of this parameter is automatically set based on hardware capacity.
Options:
1 or 2
|
Calculated at startup.
|
|
7.5.0
Update Pack 1
|
QFLOW_OVERRIDE_GARBAGE_COLLECTOR_MT_SETTINGS |
Used to override the values that are used for thread assignment for garbage collector threads.
Options:
YES = Override the default settings.
NO = Use the default settings.
|
NO |
|
7.5.0
Update Pack 1
|
QFLOW_OVERRIDE_REPORTER_MT_SETTINGS |
Used to override the default settings for the QFLOW_NUM_THREADS_REPORTER parameter.
Options:
YES = Override the default settings.
NO = Use the default settings.
|
NO |
Multi-threaded processing parameters for raw traffic
The following parameters can be used for tuning multi-threaded processing for raw network traffic.
| Version | Parameter | Description | Default value |
|---|---|---|---|
| 7.2.8 | QFLOW_RAW_TRAFFIC_MT_ON |
Specifies whether multi-threaded processing is used for raw traffic.
Options:
YES = Use multi-threaded processing.
NO = Use single-threaded processing.
|
YES |
| 7.2.8 | QFLOW_OVERRIDE_RAW_TRAFFIC_MT_SETTINGS |
Used to override the values that are used for thread assignment for raw traffic.
Options:
YES = Override the default settings.
NO = Use the default settings.
|
NO |
| 7.2.8 | QFLOW_NUM_THREADS_NIC |
Specifies the number of threads that are configured to receive input data from network interface flow sources.
By default, the value of this parameter is automatically set based on hardware capacity.
Options:
1 to 32
|
Calculated at startup. |
| 7.2.8 | QFLOW_NUM_THREADS_NAPATECH |
Specifies the number of threads that are configured to receive input data from Napatech flow sources.
There must be sufficient Napatech host buffers configured for the specified number of threads.
Options:
1, 2, 4, 8 or 10
|
Calculated at startup. |
Sequence number verification parameters
The following parameters can be used for tuning sequence number verification for flow records.
| Version | Parameter | Description | Default value |
|---|---|---|---|
|
7.5.0
Update Pack 1
|
SV_VERIFY_SEQUENCE_NUMBERS |
Specifies whether to use sequence number verification to detect when messages are dropped.
Options:
YES = Use sequence number verification.
NO = Turn sequence number verification off.
|
YES |
|
7.5.0
Update Pack 1
|
QFLOW_SEQUENCE_NUMBER_GAP_REPORT_LIMIT |
Used to specify the maximum number of gaps that are reported for a single exporter, per interval, before the gaps are truncated.
|
50 |
Document Location
Worldwide
[{"Type":"MASTER","Line of Business":{"code":"LOB24","label":"Security Software"},"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SSBQAC","label":"IBM Security QRadar SIEM"},"ARM Category":[{"code":"a8m0z000000cwsuAAA","label":"Flow Source"}],"ARM Case Number":"","Platform":[{"code":"PF016","label":"Linux"}],"Version":"7.4.1;7.4.2;7.4.3;7.5.0"}]
Was this topic helpful?
Document Information
Modified date:
21 June 2023
UID
ibm15691056