Create a custom property to extract data that IBM
QRadar does not typically
show from the event or flow payloads. Custom properties must be enabled, and extraction-based custom
properties must be parsed, before you can use them in rules, searches, reports, or for offense
indexing.
Before you begin
QRadar includes a number
of existing custom event properties that are not enabled or parsed by default. Ask your
administrator to review the custom event property that you want to create to ensure that it does not
exist.
To create custom event properties, you must have the User Defined Event
Properties permission.
To create custom flow properties, you must have the User Defined Flow
Properties permission. You must also set the IPFIX Additional Field
Encoding field to Payload or TLV and
Payload.
Users with administrative capabilities can create custom event and flow properties by selecting
Custom Event Properties or Custom Flow Properties on
the Admin tab.
You must configure a flow collector to export data to a flow processor. For more information, see
Configuring the Flow Collector format.
About this task
Although multiple default custom properties might have the same name and the same log source,
they can have different regex expressions, event names, or categories. For example, there are
multiple custom properties for Microsoft
Windows Security Event Log called
AccountName, but each one is defined by a unique regex expression.
Procedure
-
Click the Log Activity tab or the Network
Activity tab.
-
If you are viewing the events or flows in streaming mode, click the
Pause icon to pause streaming.
-
Double-click the event or flow that contains the data that you want to extract, and then click
Extract Property.
-
In the Property Type Selection pane, select the type of custom property
that you want to create.
-
Configure the custom property parameters.
Click the help icon (
) to see information about the custom property parameters.
In QRadar 7.6.0, you can
configure new parameters to create custom event properties that extract a list. Currently, the
custom event properties are supported for Regex and JSON parsing. After a property is converted to a
list, it cannot be converted back to a non-list. A list property can still parse out single values.
Flow properties cannot be a list. List properties deduplicate identical values. List properties
cannot be used in offense indexing, domain mapping, or calculated properties.
To create the custom event property, you can select the following three options for Regex List parsing:
- Parse Once
- This option is the default value for non-list.
- Parse Multiple
- This option runs the supplied expression repeatedly until there are no more matches or reach the
match limit. The match limit can be set per expression or use the global value set in system
settings.
- Parse All Capture Groups
- This option runs the supplied expression one time and extract the results of each capture group
into a list until the match limit is reached.
To create the custom event property, you can select the following two options for JSON parsing:
- Key path
- The key path points to a part of the JSON object you want to return. The key path can point to
an array of data, and returns all objects or values inside the list.
- Subfield key path
- If the key path points to an array of objects then the subfield key path can select just one
value in the objects to return as a list.
For example, key path: /”prop”/”subproperty”[] returns
[{"pid": 700},{"pid": 2232}] and Subfield path: /”pid” returns
[700,2232]
-
If you are creating an extraction-based custom property that is to be used in rules, search
indexes, or forwarding profiles, ensure that the Enable for use in Rules, Forwarding
Profiles and Search Indexing check box is selected.
- Optional: Click Test to test the expression against
the payload.
- Optional:
New in 7.5.0 Update Package 12 You can use predictive
parsing algorithm for regular expressions (regex) custom properties. You can enable predictive
parsing by selecting Enabling Predictive Parsing checkbox. By default,
predictive parsing is enabled now for all custom properties. If you enable predictive parsing, the
performance is faster when you create a new property. You can also set the delimiter set for a
property by using Predictive Parsing Delimiters option. Predictive Parsing
uses an algorithm to extract property values from events without running the regex for every event
and thus is fast. However, in rare circumstances the algorithm can make incorrect predictions, so it
is recommended to use Predictive Parsing only for log source types, which are expected to receive
high event rates and thus require the faster parsing.
-
Click Save.