IBM Support

QRadar: Log Source Extension and Multiple Match Group usage to resolve the parsing issues of source IP

Troubleshooting


Problem

At times, you might observe that on using a custom Log Source Extension (LSX) to parse logs from a specific device or application, the source IP address is not being correctly extracted and displayed in the log activity. This issue occurs despite the DSM Editor that uses a properly configured regular expression (regex) to match the IP address. Interestingly, the source IP parsing is correct when using the DSM without the LSX, indicating that the issue lies within the LSX configuration.

Symptom

  • The source IP address is correctly extracted and displayed in the DSM Editor when you use the built-in regular expression:

    (\s(\d+\.\d+\.\d+\.\d+)\%\d+\:\d+\s\-\>)

  • However, when you use a custom LSX, the source IP address is not accurately extracted and displayed in the Log Activity.

Original LSX Configuration:
The following LSX configuration uses a single match group, which fails to extract the source IP address correctly:

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<ns2:device-extension xmlns:ns2="event_parsing/device_extension">
    <pattern use-default-pattern="true" type="JavaPattern" id="EventName-Pattern-1">\s(tmm\d|tmm)\[\d+.*?(Client|Header|HTTP Host|IP|user agent).*?(asked|forwarded to pool|added to cliet|redirected to|changed to|blocked due|HTTP header Host was replaced|reset|rejected|blocked|HOST header was set|forwarded by LTM_policy|presents his client cert|no cert|tmp fwd|is fwd)</pattern>
    <pattern use-default-pattern="true" type="JavaPattern" id="EventName-Pattern-2">\s(tmm\d|tmm)\[\d+.*?\d\s(is reset by)</pattern>
    <pattern use-default-pattern="true" type="JavaPattern" id="EventName-Pattern-3">local.*?(Client).*?(authenticated)</pattern>
    <pattern use-default-pattern="true" type="JavaPattern" id="EventName-Pattern-4">(logger)\[\d+</pattern>
    <pattern use-default-pattern="true" type="JavaPattern" id="EventName-Pattern-5">(tmm|tmm\d)\[\d+\]\:\s(Rule)</pattern>
    <pattern use-default-pattern="true" type="JavaPattern" id="SourceIp-Pattern-1">from\s(\d+\.\d+\.\d+\.\d+)\%\d+\swho</pattern>
    <pattern use-default-pattern="true" type="JavaPattern" id="SourceIp-Pattern-2">[Cc]lient\s(\d+\.\d+\.\d+\.\d+)</pattern>
    <pattern use-default-pattern="true" type="JavaPattern" id="SourceIp-Pattern-3">[Cc]lient\sIP\s(\d+\.\d+\.\d+\.\d+)\%\d+</pattern>
    <pattern use-default-pattern="true" type="JavaPattern" id="SourceIp-Pattern-4">\s(\d+\.\d+\.\d+\.\d+)\%\d+\:\d+\s\-\&gt;</pattern>
    <pattern type="JavaPattern" id="AllEvents">(.*?)</pattern>
    <match-group device-type-id-override="49" order="1">
        <matcher order="1" enable-substitutions="true" capture-group="\2 \3" pattern-id="EventName-Pattern-1" field="EventName"/>
        <matcher order="2" enable-substitutions="true" capture-group="\2" pattern-id="EventName-Pattern-2" field="EventName"/>
        <matcher order="3" enable-substitutions="true" capture-group="\1 \2" pattern-id="EventName-Pattern-3" field="EventName"/>
        <matcher order="4" enable-substitutions="true" capture-group="\1" pattern-id="EventName-Pattern-4" field="EventName"/>
        <matcher order="5" enable-substitutions="true" capture-group="\2" pattern-id="EventName-Pattern-5" field="EventName"/>
        <matcher order="1" enable-substitutions="true" capture-group="\1" pattern-id="SourceIp-Pattern-1" field="SourceIp"/>
        <matcher order="2" enable-substitutions="true" capture-group="\1" pattern-id="SourceIp-Pattern-2" field="SourceIp"/>
        <matcher order="3" enable-substitutions="true" capture-group="\1" pattern-id="SourceIp-Pattern-3" field="SourceIp"/>
        <matcher order="4" enable-substitutions="true" capture-group="\1" pattern-id="SourceIp-Pattern-4" field="SourceIp"/>
        <event-match-multiple force-qidmap-lookup-on-fixup="true" send-identity="UseDSMResults" pattern-id="AllEvents"/>
    </match-group>
</ns2:device-extension>

Document Location

Worldwide

[{"Type":"MASTER","Line of Business":{"code":"LOB77","label":"Automation Platform"},"Business Unit":{"code":"BU048","label":"IBM Software"},"Product":{"code":"SSBQAC","label":"IBM Security QRadar SIEM"},"ARM Category":[{"code":"a8m0z000000cwtEAAQ","label":"Log Activity"}],"ARM Case Number":"","Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"All Versions"}]

Log InLog in to view more of this document

This document has the abstract of a technical article that is available to authorized users once you have logged on. Please use Log in button above to access the full document. After log in, if you do not have the right authorization for this document, there will be instructions on what to do next.

Document Information

Modified date:
31 March 2025

UID

ibm17229225

Page Feedback