Extension document template

The example of an extension document provides information about how to parse one particular type of Cisco FWSM so that events are not sent with an incorrect event name.

For example, if you want to resolve the word session, which is embedded in the middle of the event name:

Nov 17 09:28:26 192.0.2.1 %FWSM-session-0-302015: Built UDP connection for faddr <IP_address1>/80 gaddr <IP_address2>/31696 laddr <IP_address3>/2157 duration 0:00:00 bytes 57498 (TCP FINs)

This condition causes the DSM to not recognize any events and all the events are unparsed and associated with the generic logger.

Although only a portion of the text string (302015) is used for the QID search, the entire text string (%FWSM-session-0-302015) identifies the event as coming from a Cisco FWSM. Since the entire text string is not valid, the DSM assumes that the event is not valid.

Extension document example for parsing one event type

An FWSM device has many event types and many with unique formats. The following extension document example indicates how to parse one event type.

Note: The pattern IDs do not have to match the field names that they are parsing. Although the following example duplicates the pattern, the SourceIp field and the SourceIpPreNAT field cab use the exact same pattern in this case. This situation might not be true in all FWSM events.

<?xml version="1.0" encoding="UTF-8"?>
<device-extension xmlns="event_parsing/device_extension"> 
<pattern id="EventNameFWSM_Pattern" xmlns=""><![CDATA[%FWSM[a-zA-Z\-]*\d-(\d{1,6})]]></pattern>
<pattern id="SourceIp_Pattern" xmlns=""><![CDATA[gaddr (\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})/([\d]{1,5})]]></pattern> 
<pattern id="SourceIpPreNAT_Pattern" xmlns=""><![CDATA[gaddr (\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})/([\d]{1,5})]]></pattern>
<pattern id="SourceIpPostNAT_Pattern" xmlns=""><![CDATA[laddr (\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})/([\d]{1,5})]]></pattern>
<pattern id="DestinationIp_Pattern" xmlns=""><![CDATA[faddr (\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})/([\d]{1,5})]]></pattern>
<pattern id="Protocol_Pattern" case-insensitive="true" xmlns=""><![CDATA[(tcp|udp|icmp|gre)]]></pattern>
<pattern id="Protocol_6_Pattern" case-insensitive="true" xmlns=""><![CDATA[protocol=6]]></pattern> 
<pattern id="EventNameId_Pattern" xmlns=""><![CDATA[(\d{1,6})]]></pattern>
<match-group order="1" description="FWSM Test" device-type-id-override="6" xmlns=""> 
      <matcher field="EventName" order="1" pattern-id="EventNameFWSM_Pattern" capture-group="1"/>     
      <matcher field="SourceIp" order="1" pattern-id="SourceIp_Pattern" capture-group="1" /> 
      <matcher field="SourcePort" order="1" pattern-id="SourcePort_Pattern" capture-group="2"/> 
      <matcher field="SourceIpPreNAT" order="1" pattern-id="SourceIpPreNAT_Pattern" capture-group="1" /> 
      <matcher field="SourceIpPostNAT" order="1" pattern-id="SourceIpPostNAT_Pattern" capture-group="1" /> 
      <matcher field="SourcePortPreNAT" order="1" pattern-id="SourcePortPreNAT_Pattern" capture-group="2" />   
      <matcher field="SourcePortPostNAT" order="1" pattern-id="SourcePortPostNAT_Pattern" capture-group="2" /> 
      <matcher field="DestinationIp" order="1" pattern-id="DestinationIp_Pattern" capture-group="1" />         
      <matcher field="DestinationPort" order="1" pattern-id="DestinationIp_Pattern" capture-group="2" /> 
      <matcher field="Protocol" order="1" pattern-id="Protocol_Pattern" capture-group="1" /> 
      <matcher field="Protocol" order="2" pattern-id="Protocol_6_Pattern" capture-group="TCP" enable-substitutions=true/>      
      <event-match-multiple pattern-id="EventNameId" capture-group-index="1" device-event-category="Cisco Firewall"/> 
   </match-group> 
</device-extension>

<?xml version="1.0" encoding="UTF-8"?> 
<device-extension xmlns="event_parsing/device_extension"> 
<!-- Do not remove the "allEventNames" value -->
<pattern id="EventName-Fakeware_Pattern" xmlns=""><![CDATA[]]></pattern>
<pattern id="SourceIp-Fakeware_Pattern" xmlns=""><![CDATA[]]</pattern>
<pattern id="SourcePort-Fakeware_Pattern" xmlns=""><![CDATA[]]></pattern>
<pattern id="SourceMAC-Fakeware_Pattern" xmlns=""><![CDATA[]]></pattern>
<pattern id="DestinationIp-Fakeware_Pattern" xmlns=""><![CDATA[]]></pattern>
<pattern id="DestinationPort-Fakeware_Pattern" case-insensitive="true" xmlns=""><![CDATA[]]></pattern>
<pattern id="Protocol-Fakeware_Pattern" case-insensitive="true" xmlns=""><![CDATA[]]></pattern> 
<match-group order="1" description="FWSM Test" device-type-id-override="6" xmlns=""> 
   <matcher field="EventName" order="1" pattern-id="EventName-Fakeware_Pattern" capture-group="1"/> 
   <matcher field="SourceIp" order="1" pattern-id="SourceIp-Fakeware_Pattern" capture-group="1" /> 
   <matcher field="SourcePort" order="1" pattern-id="SourcePort-Fakeware_Pattern" capture-group="1"/> 
   <matcher field="SourceMAC" order="1" pattern-id="SourceMAC-Fakeware_Pattern" capture-group="1" /> 
   <matcher field="DestinationIp" order="1" pattern-id="DestinationIp-Fakeware_Pattern" capture-group="1" /> 
   <matcher field="DestinationPort" order="1" pattern-id="SDestinationPort-Fakeware_Pattern" capture-group="1" /> 
   <matcher field="Protocol" order="1" pattern-id="Protocol-Fakeware_Pattern" capture-group="1" />   
   <event-match-multiple pattern-id="EventNameId" capture-group-index="1" device-event-category="Cisco Firewall"/> 
</match-group> 
</device-extension>

Parsing basics

The preceding extension document example demonstrates some of the basic aspects of parsing:

IP addresses
Ports
Protocol
Multiple fields that use the same pattern with different groups

This example parses all FWSM events that follow the specified pattern. The fields that are parsed might not be present in those events when the events include different content.

The information that was necessary to create this configuration that was not available from the event:

The event name is only the last 6 digits (302015) of the %FWSM-session-0-302015 portion of the event.
The FWSM has a hardcoded device event category of Cisco Firewall.
The FWSM DSM uses the Cisco Pix QIDmap and therefore includes the device-type-id-override="6" parameter in the match group. The Pix firewall log source type ID is 6.

Note: If the QID information is not specified or is unavailable, you can modify the event mapping. For more information, see the Modifying Event Mapping section in the IBM® QRadar User Guide.

Event name and device event category

An event name and a device event category are required when the QIDmap is searched. This device event category is a grouping parameter within the database that helps define like events within a device. The event-match-multiple at the end of the match group includes hardcoding of the category. The event-match-multiple uses the EventNameId pattern on the parsed event name to match up to 6 digits. This pattern is not run against the full payload, just that portion parsed as the EventName field.

The EventName pattern references the %FWSM portion of the events; all Cisco FWSM events contain the %FWSM portion. The pattern in the example matches %FWSM followed by any number (zero or more) of letters and dashes. This pattern match resolves the word session that is embedded in the middle of the event name that needs to be removed. The event severity (according to Cisco), followed by a dash and then the true event name as expected by QRadar®. The (\d{6}) string is the only string within the EventNameFWSM pattern that has a capture group.

The IP addresses and ports for the event all follow the same basic pattern: an IP address followed by a colon followed by the port number. This pattern parses two pieces of data (the IP address and the port), and specifies different capture groups in the matcher section.

<device-extension>
<pattern id="EventName1">(logger):</pattern> 
<pattern id="DeviceTime1">time=\[(\d{2}/\w{3}/\d{4}:\d{2}:\d{2}:\d{2})\] </pattern>
<pattern id="Username">(TLSv1)</pattern> 
<match-group order="1" description="Full Test"> 
   <matcher field="EventName" order="1" pattern-id="EventName1" capture-group="1"/>   
   <matcher field="DeviceTime" order="1" pattern-id="DeviceTime1"  
       capture-group="1" ext-data="dd/MMM/YYYY:hh:mm:ss"/> 
   <matcher field="UserName" order="1" pattern-id="Username" capture-group="1"/>
</match-group>
</device-extension>

IP address and port patterns

The IP address and port patterns are four sets of one to three digits, separated by periods followed by a colon and the port number. The IP address section is in a group, as is the port number, but not the colon. The matcher sections for these fields reference the same pattern name, but a different capture group (the IP address is group 1 and the port is group 2).

The protocol is a common pattern that searches the payload for the first instance of TCP, UDP, ICMP, or GRE. The pattern is marked with the case-insensitive parameter so that any occurrence matches.

Although a second protocol pattern does not occur in the event that is used in the example, there is a second protocol pattern that is defined with an order of two. If the lowest-ordered protocol pattern does not match, the next one is attempted, and so on. The second protocol pattern also demonstrates direct substitution; there are no match groups in the pattern, but with the enable-substitutions parameter enabled, the text TCP can be used in place of protocol=6.