IBM Support

QRadar: How do I convert epoch time to use in my DSM

Troubleshooting


Problem

My Log source has epoch time in the payload. Is there a way to get the DSM to convert this properly?

Resolving The Problem

Some log sources use epoch time in their payload. The epoch time becomes an issue for most DSMs because the Log Source Extension framework tries to inject a YEAR field into timestamps that do not already contain a year. The problem is that epoch timestamps would not contain a YEAR field in the format. The addition of the YEAR field causes the parse of the epoch timestamp to be wrong. The way to remedy this situation is to force the log source extension to use a Year field by putting 'yyyy' in single quotes within the date format present in the timestamp. Characters encapsulated in single quotes within a date format is NOT interpreted by the date formatter, so they are ignored. This results in the epoch timestamp being parsed.
This is an example of what the payload looks like, with log source time being an epoch value.
event=event1 cat=cat1 log_source_time=1506882631123 username=Example field=field1
 
To resolve this issue.
  1. Log in to the QRadar UI
  2. Click the Admin tab.
  3. Scroll to Data Sources > click DSM Editor.
  4. Locate your DSM, where you are receiving payloads with epoch time.
  5. Under Properties type Log Source Time in the Filter search.

    image-20190924124212-4
     
  6. Click the checkbox Override system behavior.
  7. In the field expression type: log_source_time=(.*?)\s
  8. In the field format string type: $1yyyy
  9. In the field date format type:ssssssssssSSS'yyyy'
  10. Click OK.
  11. Click Save.
Results
Your Log Source now parses events with epoch time in the payload.

Related Information

Document Location

Worldwide

[{"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SSBQAC","label":"IBM Security QRadar SIEM"},"Component":"DSM;DSM editor;Parsing;","Platform":[{"code":"PF016","label":"Linux"}],"Version":"All Versions","Edition":"","Line of Business":{"code":"LOB24","label":"Security Software"}}]

Document Information

Modified date:
08 January 2021

UID

ibm10886907