Topic
  • 11 replies
  • Latest Post - ‏2015-03-12T19:48:38Z by hclewelad
grol
grol
36 Posts

Pinned topic How to manage xml captured in probe

‏2013-10-31T11:43:09Z |

Hi,

Are there a way to manage an export probe?.

I want to make date searches or add filters. A tool that allow handle captured messages like you can do in wireshark for example..

 

Regards,

Grol

  • HermannSW
    HermannSW
    6126 Posts

    Re: How to manage xml captured in probe

    ‏2013-11-01T17:40:37Z  

    Hi,

    Probe capture exports are only intended to be imported into the same or another DataPower box in order to be viewed by Probe.

    This posting explains the Probe capture structure and how you can view any (even Non-XML) context contents in <transaction-input> element:
    https://www.ibm.com/developerworks/community/blogs/HermannSW/entry/wanna_know_what_is_in_probe_context_displayed_empty

    There are no Probe capture tools.


    Hermann<myXsltBlog/> <myXsltTweets/> <myCE/>

     

  • grol
    grol
    36 Posts

    Re: How to manage xml captured in probe

    ‏2013-11-04T09:13:49Z  
    • HermannSW
    • ‏2013-11-01T17:40:37Z

    Hi,

    Probe capture exports are only intended to be imported into the same or another DataPower box in order to be viewed by Probe.

    This posting explains the Probe capture structure and how you can view any (even Non-XML) context contents in <transaction-input> element:
    https://www.ibm.com/developerworks/community/blogs/HermannSW/entry/wanna_know_what_is_in_probe_context_displayed_empty

    There are no Probe capture tools.


    Hermann<myXsltBlog/> <myXsltTweets/> <myCE/>

     

    Thanks for the response.

    Then If you have a service with a lot of traffic and you are tracing it, how you can do to find one request in particular?

     

    Regards,

    Grol.

  • HermannSW
    HermannSW
    6126 Posts

    Re: How to manage xml captured in probe

    ‏2013-11-05T09:17:17Z  
    • grol
    • ‏2013-11-04T09:13:49Z

    Thanks for the response.

    Then If you have a service with a lot of traffic and you are tracing it, how you can do to find one request in particular?

     

    Regards,

    Grol.

    First let me restate the most important message on Probe from  InfoCenter:
    http://pic.dhe.ibm.com/infocenter/wsdatap/v5r0m0/index.jsp?topic=%2Fcom.ibm.dp.doc%2Fproblemdetermination73.htm

    ...
    CAUTION:
    Do not leave the probe in the enabled state for production conditions. Disable the probe before moving the configuration into production.
    ...


    So we are talking on develeopment or test Probing here.

    One way to restrict the transactions that actually are captured is by using Probe Triggeres.
    From WebGUI help:

     

    Probe Triggers

    The probe captures transactions that meet one or more of the conditions defined by the triggers. These triggers examine the direction or type of the message flow and examine the message for an XPath expression match. When a message meets one of these conditions, the transaction is captured in diagnostics mode and becomes part of the list of transactions that can be viewed.

    Client-IP Match

    Specify a PCRE to match against client IP addresses. Requests from clients with matching IP addresses will trigger the probe. To create a match for all IP addresses, specify .* instead of * as the PCRE.

    Inbound URL Match

    Specify a PCRE to match against the inbound URLs. Requests from clients with matching URLs will trigger the probe. To create a match for all URLs, specify .* instead of * as the PCRE.

    Outbound URL Match

    Specify a PCRE to match against the outbound URLs. Responses from servers with matching URLs will trigger the probe. To create a match for all URLs, specify .* instead of * as the PCRE.

    Processing Type

    Select the rule direction or type that will trigger the probe.

    • Any
    • Response

      Response rule (server-originated)

    • Request

      Request rule (client-originated)

    • Call

      Call processing rule

    • Error

      Error rule

    Processing Rule Match

    Specify a PCRE to match against names of processing rules. Rules with matching names will trigger the probe. To create a match for all names, specify .* instead of * as the PCRE.

    XPath Expression Match

    Specify an XPath expression of use the XPath Tool to define an XPath expression to match against messages. Messages that contain the expression will trigger the probe.



    Hermann<myXsltBlog/> <myXsltTweets/> <myCE/>

     

  • grol
    grol
    36 Posts

    Re: How to manage xml captured in probe

    ‏2013-11-12T12:02:49Z  
    • HermannSW
    • ‏2013-11-05T09:17:17Z

    First let me restate the most important message on Probe from  InfoCenter:
    http://pic.dhe.ibm.com/infocenter/wsdatap/v5r0m0/index.jsp?topic=%2Fcom.ibm.dp.doc%2Fproblemdetermination73.htm

    ...
    CAUTION:
    Do not leave the probe in the enabled state for production conditions. Disable the probe before moving the configuration into production.
    ...


    So we are talking on develeopment or test Probing here.

    One way to restrict the transactions that actually are captured is by using Probe Triggeres.
    From WebGUI help:

     

    Probe Triggers

    The probe captures transactions that meet one or more of the conditions defined by the triggers. These triggers examine the direction or type of the message flow and examine the message for an XPath expression match. When a message meets one of these conditions, the transaction is captured in diagnostics mode and becomes part of the list of transactions that can be viewed.

    Client-IP Match

    Specify a PCRE to match against client IP addresses. Requests from clients with matching IP addresses will trigger the probe. To create a match for all IP addresses, specify .* instead of * as the PCRE.

    Inbound URL Match

    Specify a PCRE to match against the inbound URLs. Requests from clients with matching URLs will trigger the probe. To create a match for all URLs, specify .* instead of * as the PCRE.

    Outbound URL Match

    Specify a PCRE to match against the outbound URLs. Responses from servers with matching URLs will trigger the probe. To create a match for all URLs, specify .* instead of * as the PCRE.

    Processing Type

    Select the rule direction or type that will trigger the probe.

    • Any
    • Response

      Response rule (server-originated)

    • Request

      Request rule (client-originated)

    • Call

      Call processing rule

    • Error

      Error rule

    Processing Rule Match

    Specify a PCRE to match against names of processing rules. Rules with matching names will trigger the probe. To create a match for all names, specify .* instead of * as the PCRE.

    XPath Expression Match

    Specify an XPath expression of use the XPath Tool to define an XPath expression to match against messages. Messages that contain the expression will trigger the probe.



    Hermann<myXsltBlog/> <myXsltTweets/> <myCE/>

     

    Sorry for the delay.

    I have been reading about how to logging request and response payloads. That's what we need. Triggers is not the solution.

    In one post someboy talks about to log this sending asynchronous messages to MQ and then insert them into a database. We don't have MQ but we have TIBCO EMS.

    Is this solution good for PRO enviroments?

    I'm not sure how to send request to EMS. I guess that you have to add a extra result action in the ws policy to route the messages. Is that correct? 

    Thank you

    Regards.

    Guillermo.

  • kenhygh
    kenhygh
    2059 Posts

    Re: How to manage xml captured in probe

    ‏2013-11-12T18:02:01Z  
    • grol
    • ‏2013-11-12T12:02:49Z

    Sorry for the delay.

    I have been reading about how to logging request and response payloads. That's what we need. Triggers is not the solution.

    In one post someboy talks about to log this sending asynchronous messages to MQ and then insert them into a database. We don't have MQ but we have TIBCO EMS.

    Is this solution good for PRO enviroments?

    I'm not sure how to send request to EMS. I guess that you have to add a extra result action in the ws policy to route the messages. Is that correct? 

    Thank you

    Regards.

    Guillermo.

    Guillermo,

    Check the InfoCenter for dp:url-open() for EMS endpoints to see how to specify the details if you want. Or, better, use an asynchronous-results action.

  • tallurisri
    tallurisri
    105 Posts

    Re: How to manage xml captured in probe

    ‏2013-11-12T19:21:43Z  
    • grol
    • ‏2013-11-12T12:02:49Z

    Sorry for the delay.

    I have been reading about how to logging request and response payloads. That's what we need. Triggers is not the solution.

    In one post someboy talks about to log this sending asynchronous messages to MQ and then insert them into a database. We don't have MQ but we have TIBCO EMS.

    Is this solution good for PRO enviroments?

    I'm not sure how to send request to EMS. I guess that you have to add a extra result action in the ws policy to route the messages. Is that correct? 

    Thank you

    Regards.

    Guillermo.

    Hi Grol,

    we are doing this from transform action by using xsl file with dp:url-open to ftp server. 

     

    Thank you,

    Sri.

     

     

     

  • hclewelad
    hclewelad
    27 Posts

    Re: How to manage xml captured in probe

    ‏2015-03-11T18:15:23Z  

    I am having the same issue trying to capture the output from a non-xml backend. The input for this is all in the URL but the backend service is returning an HTML based error message that I need to log in a probe on an Probe Error Trigger.
     

    Could the fact that the backend is SSL affect the use of a Probe?

    Thanks,

    dale

  • kenhygh
    kenhygh
    2059 Posts

    Re: How to manage xml captured in probe

    ‏2015-03-11T19:12:52Z  
    • hclewelad
    • ‏2015-03-11T18:15:23Z

    I am having the same issue trying to capture the output from a non-xml backend. The input for this is all in the URL but the backend service is returning an HTML based error message that I need to log in a probe on an Probe Error Trigger.
     

    Could the fact that the backend is SSL affect the use of a Probe?

    Thanks,

    dale

    Dale,

    No, by the time the data gets to the probe it's unencrypted.

    Capturing this in the probe is probably not the best idea, you do not want probe on in Production. Is logging the content sufficient?

  • hclewelad
    hclewelad
    27 Posts

    Re: How to manage xml captured in probe

    ‏2015-03-11T19:38:30Z  
    • kenhygh
    • ‏2015-03-11T19:12:52Z

    Dale,

    No, by the time the data gets to the probe it's unencrypted.

    Capturing this in the probe is probably not the best idea, you do not want probe on in Production. Is logging the content sufficient?

    Yeah. I think that's the best thing too. I already have a xsl that just logs a debug message but since the response is txt I am missing something basic about capturing the response.

    This should work but it's not showing the response from the backend:

    <xsl:template match="/">
      <xsl:message dp:priority="debug" ><xsl:value-of select="' This is just a test:' " />
      <xsl:copy-of select="." />
      </xsl:message>
     </xsl:template>

    The log shows:

    mpgw (Dales_REST_Gateway): This is just a test:

    Thanks,

    dale

  • HermannSW
    HermannSW
    6126 Posts

    Re: How to manage xml captured in probe

    ‏2015-03-12T07:05:48Z  
    • hclewelad
    • ‏2015-03-11T19:38:30Z

    Yeah. I think that's the best thing too. I already have a xsl that just logs a debug message but since the response is txt I am missing something basic about capturing the response.

    This should work but it's not showing the response from the backend:

    <xsl:template match="/">
      <xsl:message dp:priority="debug" ><xsl:value-of select="' This is just a test:' " />
      <xsl:copy-of select="." />
      </xsl:message>
     </xsl:template>

    The log shows:

    mpgw (Dales_REST_Gateway): This is just a test:

    Thanks,

    dale

    Hi Dale,

    for Non-XML response you need a Non-XML stylesheet (DataPower proprietary, other XSLT processors can only deal with XML).

    Please find the information here:


    Hermann.

  • hclewelad
    hclewelad
    27 Posts

    Re: How to manage xml captured in probe

    ‏2015-03-12T19:48:38Z  

     Hermann FTW again...

    The solution involved listening to the presentations above and then creating a String FFD based handler

    The XSL that logs looks like this now:
     

    <xsl:stylesheet version="1.0"
      xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
      xmlns:dp="http://www.datapower.com/extensions"
      extension-element-prefixes="dp"
    >
      <dp:input-mapping href="String.utf-8.ffd" type="ffd"/>
     
      <xsl:output omit-xml-declaration="yes" method="text" />

      <xsl:template match="/">
       <xsl:message dp:priority="debug" ><xsl:value-of select="' This is just a test:' " />
         <xsl:copy-of select="/object/message"/>
        </xsl:message>
        <xsl:copy-of select="/object/message"/>
      </xsl:template>
    </xsl:stylesheet>

    I just realized how incredibly off the original topic this is, but since it's already done we might as let google see it.

    dale