Topic
3 replies Latest Post - ‏2013-03-14T16:24:27Z by SystemAdmin
SystemAdmin
SystemAdmin
483 Posts
ACCEPTED ANSWER

Pinned topic Data from Universal Feed Sample not populated in reports

‏2013-03-14T08:19:33Z |
I am interested in learning more about the Universal Feed to Guardium. So, I built and executed the sample supplied in Joe DiPetro's part 1 tutorial and found fields were not being populated in the reports such as

From the Accessor message
comm_protocol: "CommProtocol"
db_protocol: "DB Protocol"
db_protocol_version: "DBProtocolVersion"

In the CLIENT_REQUEST message I only ever see the full sql. I never get the SQL with ? marks replacing the bind variables. Is there something special that needs to be done in SQLGuard to get these fields to populate?

Also, the datasource.java class defines a number of other message types. Some are simple and some are more complex. Is there any documentation available to explain when and how to use these interfaces?

Thanks
Michail
Updated on 2013-03-14T16:24:27Z at 2013-03-14T16:24:27Z by SystemAdmin
  • SystemAdmin
    SystemAdmin
    483 Posts
    ACCEPTED ANSWER

    Re: Data from Universal Feed Sample not populated in reports

    ‏2013-03-14T12:33:04Z  in response to SystemAdmin
    Hi Michail,

    I've been working on an STAP for mongoDB that you might be interested in. You can find it here:
    https://github.com/johnhaldeman/mongoTap

    I have found similar things to your experience. Some things like Comm Protocol, DB Protocol, and DBProtocolVersion are hit and miss depending on the language type chosen. I haven't seen any formal documentation and would be interested in the same :)

    I have not implemented or tested, but I think the SQL with ? markers may have to be populated directly by your agent using the original_sql field of GDM_construct.

    I also find Datasource.java hard to read and wanted to use Ruby, so I have been extracting the protocol buffer definition (the *.pb file) out of the compiled results in Datasource.java. It's not complete yet, but this is what I've come up with so far in order to get the mongoTap to work:
    https://github.com/johnhaldeman/mongoTap/blob/master/mongoTap/Datasource.proto

    Hope that helps! Sorry I couldn't be more definitive.

    Cheers,

    John
    • SystemAdmin
      SystemAdmin
      483 Posts
      ACCEPTED ANSWER

      Re: Data from Universal Feed Sample not populated in reports

      ‏2013-03-14T14:08:47Z  in response to SystemAdmin
      Hi John,

      This is interesting to learn more about the Universal Feed. May I know is the Universal Feed only work on Unix/Linux platform? Does it work on Windows?

      Do we need to write the script from scratch? What tools or language can we used?
      regards,
      Teh
      • SystemAdmin
        SystemAdmin
        483 Posts
        ACCEPTED ANSWER

        Re: Data from Universal Feed Sample not populated in reports

        ‏2013-03-14T16:24:27Z  in response to SystemAdmin
        Hi Teh,

        We're probably hijacking this thread inappropriately, but the universal feed is really just a protocol you can implement. As such it is definitely not unix/linux only (the mongoTap, however, is linux/unix only because it relies on mongosniff which in turn relies on pcap - but this dependency for the mongoTap has nothing to do with the universal feed).

        For details on the protocol, these are the best resources:

        http://www.ibm.com/developerworks/data/library/techarticle/dm-1210universalfeed/index.html

        http://www.ibm.com/developerworks/data/library/techarticle/dm-1211universalfeed2/index.html
        You will likely have to code something to implement the protocol if you want to use the universal feed. In general you need the following:
        1) A source of information to forward to Guardium. The mongoTap uses mongoDB's mongosniff utility for this. In Joe's article above, the source of data is a text file
        2) A translation mechanism to receive that feed of data and translate it to the universal feed protocol describe in the article. For the mongoTap, this is the mongoTap Server component

        As for programming languages, because the universal feed is just a protocol you implement, you can use any language. That being said, you probably want to use a language that has a protocol buffer (protobuf) library so that you don't have to reimplement protocol buffers yourself :). Probably the most developed protobuf libraries are the ones for Java, Python, and C++. Those are open source libraries maintained by Google. This doesn't mean you can't use other languages. For instance, Ruby's protocol buffer library implements more than enough of the specification to support the universal feed protocol. That's why the mongoTap could be programmed in Ruby.

        Cheers,

        John