QRadar: How to change or customize Log Source Time

How To

Summary

This article is intended to help customize the time that is extracted by QRadar® for the Log Source Time.

Objective

Administrators can use the DSM editor to change the extracted Log Source Time, but an understanding of QRadars date format is required.

Environment

Date Format:

The date can be represented in different ways, for example, a date can be present in the event by using format 2020-12-31 or this other format 12/31/2020, or in epoch time 1594390116000, or in some cases this information could be distributed on the payload. For example, the date could be in one part and the time in a different part of the payload.

Steps

These examples demonstrate how Log Source Times can be customized:

Example	Date	Regex	Date Format	Format string
Date Time without year (Situation1)	Jul 15 10:53:58	(\w{3} \d{1,2} \d{1,2}:\d{2}:\d{2})	MMM dd HH:mm:ss	$1
Date Time with year (Situation 1)	25-OCT-2017 03:30:54	(\d{1,2}-\w{3}-\d\d\d\d \d{1,2}:\d{1,2}:\d{1,2})	DD-MMM-yyyy HH:mm:ss	$1 $2
Epoch conversion (Situation 2)	start=1594390116032	start=(.*?)\s	sssssssssSSS	$1
Date and time split in payload (Situation 3)	date=1015 time=10:53:56	date=(.?)\stime=(.?)\s	MMddHH:mm:ss	$1$2

Situation 1: Use of a custom Log Source Type to find the Log Source Time.

By default when you create a Log Source Type the Log Source Time is the time when the event is processed. If an administrator requires a customized Log Source Time, here is a method to change the default Log Source Time by overwriting it using the DSM Editor.

Locate the Log Source you need to change the Time Date.
Click Override system behavior.
Locate the date field in the Payload.
Select Expression type Regex.
Create an Expression. For the example, we are using (\D\w\w \d{1,2}:\d{1,2}:\d{1,2})
Create a Date Format to match the payload. For our example, the format is MMM dd HH:mm:ss
For the Format String, use $1
Click OK.
Click Save.

Note:

Though the Format String field asks only you for the group number: "1" use "$1" instead.
Add spaces and special characters (":", "-", ".") if they exist on the date.
If DSM is not matching your regex and you know it is correct, confirm the option named "Use Predictive Parsing" is cleared.
If you use a Date Time with a year, the format string is $1$2

Situation 2: Payload uses epoch time.

If your payload contains the time in epoch time, the process is similar. Use a regex to locate where the epoch time is in the payload, in this case, epoch time is after the string "start="

Locate the Log Source you need to change the Time Date.
Click Override system behavior.
Locate the date field in the Payload.
Select Expression type Regex.
Create an Expression. For example, we are using start=(.*?)\s
Create a Date Format to match the payload. For our example, the format is sssssssssSSS
For the Format String use $1
Click OK.
Click Save.

Note: We add ten "s", which tells QRadar that ten "s" are epoch time. In the example, epoch time uses 13 characters. The last three are for the milliseconds and three "S" are added at the end of the Date Format. If your epoch time has only 10 characters, then only add the "s".

Situation 3: Date and time are distributed in the payload.

In some cases, the time and the data could be in different sections of the payload:

In the example, two groups are used. Each group is a section of the regex that we mark with parenthesis "()". If you look at the Regex, you notice that two groups are used. The groups are counted from left to right. The first group is capturing what between "date=" and space "\s". The second is capturing what is between "time=" and space "\s". The Format String is designated by using $1$2, which tells QRadar to use both groups.

Locate the Log Source you need to change the Time Date.
Click Override system behavior.
Locate the date field in the Payload.
Select Expression type Regex.
Create an Expression. For the example, we are using date-(.?)/s.*time=(.*?)\s
Create a Date Format to match the payload. For our example, the format is MMddHH:mm:ss
For the Format String, use $1$2
Click OK.
Click Save.

In Date Format search for "101510:53:58" which is month "10", day "15", and the time in "hour:minute:second" format "10:53:58".

Results
These examples can be used as a reference for parsing dates and times. After you click Save, these changes are applied only to the Log Source Type you selected in the DSM Editor. You can see what is the Log Source Type you are editing at the top-left part of the DSM Editor.

Additional Information

Related Information

LEEF Custom event date format

AQL Date Time Formats

Document Location

Worldwide

[{"Line of Business":{"code":"LOB24","label":"Security Software"},"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SSBQAC","label":"IBM Security QRadar SIEM"},"ARM Category":[{"code":"a8m0z000000cwt0AAA","label":"Log Source"}],"ARM Case Number":"TS003057689","Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"All Version(s)"}]

Tips