Topic
IC4NOTICE: developerWorks Community will be offline May 29-30, 2015 while we upgrade to the latest version of IBM Connections. For more information, read our upgrade FAQ.
7 replies Latest Post - ‏2014-06-12T21:16:30Z by RoseD
aberdamy
aberdamy
5 Posts
ACCEPTED ANSWER

Pinned topic How to get original DSM's?

‏2013-04-24T18:11:37Z |

Is there a way to view the regular expression syntax for the DSM's that come with QRadar?

  • Aaron_Breen(IBM)
    Aaron_Breen(IBM)
    89 Posts
    ACCEPTED ANSWER

    Re: How to get original DSM's?

    ‏2013-04-24T19:24:55Z  in response to aberdamy

    We do not expose those by default :( If there is something in particular you are looking for, I may be able to look them up (specific event type)

    • aberdamy
      aberdamy
      5 Posts
      ACCEPTED ANSWER

      Re: How to get original DSM's?

      ‏2013-04-24T19:35:35Z  in response to Aaron_Breen(IBM)

      I would like to see the Apache HTTP and IpTables if possible.

       

      thanks

      • Aaron_Breen(IBM)
        Aaron_Breen(IBM)
        89 Posts
        ACCEPTED ANSWER

        Re: How to get original DSM's?

        ‏2013-04-24T20:02:12Z  in response to aberdamy

        we would not expose all regex for the entire DSM. Coming up with all the patterns is part of the IP of the product. If you have a particular event pattern, then I can look into. Also, what is the use case for this? Are you looking to see what we do parse (which would be a different request)

        • aberdamy
          aberdamy
          5 Posts
          ACCEPTED ANSWER

          Re: How to get original DSM's?

          ‏2013-04-24T20:47:14Z  in response to Aaron_Breen(IBM)

          Ok, I understand.  I would like to see what the Linux OS DSM is parsing out.  Looks like Q1 is using open pattern match to iptables because some Linux  versions simply consider it a kernel event with no other qualifier.  Our Linux is all Redhat which includes "iptables:" always so we could match more specifically.  So we are running into the issue of whether we create one big Custom Extension for the Linux OS DSM that will pattern match every relevant event coming from our Red Hat servers.

          thanks

          • Jeff Rusk (IBM)
            Jeff Rusk (IBM)
            21 Posts
            ACCEPTED ANSWER

            Re: How to get original DSM's?

            ‏2013-04-25T11:42:57Z  in response to aberdamy

            Hi, I'm one of the QRadar Integration Team developers involved in maintaining the DSMs.   Aaron is correct that we don't expose all the regular expressions from the DSMs (and actually not all the parsing is even based on simple regular expressions as, due to performance requirements, some parsing has been optimized through other functions and these aren't easy to translate back to regular expressions).

            Its difficult to offer a definitive explanation without looking at any event payloads though.  Have you tried using the Linux IP Tables DSM above the Linux OS DSM in the parsing order?   Linux OS (and actually a number of the Unix-OS-like DSMs) are best at the bottom of any parsing order due to their likelihood of parsing pretty much everything that is formatted similar to OS-like syslog messages. 

            There are a number of common "OS Services" modules that are chained onto the end of parsing of certain DSMs like the Linux OS DSM to parse all those fairly generic OS messages (cron, login, ssh, etc.).  The IP Tables DSM though does not do this so it should be ahead of the Linux OS DSM in the parsing order (the Apache DSM should also be put ahead of the Linux OS DSM in the parsing order).

            If you find that using the IP Tables DSM is still parsing non-IP Tables logs that the Linux OS DSM should instead be parsing, that would likely be an issue you could log with support (and I could keep an eye out for it and have a look at it) and would be easy to correct in a subsequent release of that DSM.

            Jeff

            • aberdamy
              aberdamy
              5 Posts
              ACCEPTED ANSWER

              Re: How to get original DSM's?

              ‏2013-04-26T14:12:12Z  in response to Jeff Rusk (IBM)

              Thanks for the reply.  We have played with the parsing order and are ordering the log sources in terms of specificity for Apache, ipTables and LinuxOS, however we found that we had to delete all these log sources that were automatically discovered and add them one by one.  I don't think we will want to play with the parsing order going forward via trial and error because these systems will more than just these 3 log source types.  We will probably wind up creating a custom DSM  for Linux with multiple pattern id matches for the various things we're looking for.

              • RoseD
                RoseD
                28 Posts
                ACCEPTED ANSWER

                Re: How to get original DSM's?

                ‏2014-06-12T21:16:30Z  in response to aberdamy

                Hi aberdamy - I'm interested in what you determined.  Unless, I'm reading this incorrectly, it is discouraging that we would need to consume 3 log sources (Apache, ipTables and LinuxOS)  not including any other application hosted on the OS for a single host since licenses are partially based on log sources.  I would think that a single combined DSM would be able to handle this and avoid tripling my license consumption for a single RHEL server.