Topic
  • 17 replies
  • Latest Post - ‏2013-12-09T22:28:30Z by Nik_DPDev
tallurisri
tallurisri
103 Posts

Pinned topic How to parse html content on response rule.

‏2013-10-09T19:36:56Z |

Hi,

We are using MPGW to create restful service. Client uses HTTP POST method to request information from 3rd party application, request and response will be in XML format and client like to see response codes form server so we enable "Process back end errors" and "Proxy HTTP Response" options with in MPGW.

Service working fine if response code is 200 with XMl body, but it failing if response code is either 404/500/401 because the error response body is in html format, DP not parsing html body and invoking error rule and sending error message to client with response code 500.

The response type Iam using in MPGW is "XML", I can't use res'pass-through" response type.   We are developing this service on XI52 v6.

Is there any way to parse html body on response rule with out changing response type from XML ?

Please share your valuable thoughts.

Thanks,

Sri.

 

 

  

  • kenhygh
    kenhygh
    1576 Posts

    Re: How to parse html content on response rule.

    ‏2013-10-10T13:55:07Z  

    DataPower parses well-formed XML, not HTML. Can you use non-XML for your response, and then key off of the response code to determine whether to parse or not?

  • tallurisri
    tallurisri
    103 Posts

    Re: How to parse html content on response rule.

    ‏2013-10-10T20:25:26Z  
    • kenhygh
    • ‏2013-10-10T13:55:07Z

    DataPower parses well-formed XML, not HTML. Can you use non-XML for your response, and then key off of the response code to determine whether to parse or not?

    Hi Kenhygh,

     

    Thanks for your reply, I tried with response type non-xml, same error.

    mpgw (TES_Inbound_MPG)Parse error

    mpgw (TES_Inbound_MPG)rule (TES_Routing_Policy_response): implied action Parse input as XML failed: illegal character 'C' at offset 0 of 

     

    Back end server responding with response code 500 and error message body in html format in this case.

    Is there any way to parse html when response type will be XML/ Non-XML?

     

    Thanks,

    Sri. 

  • Hemchan
    Hemchan
    21 Posts

    Re: How to parse html content on response rule.

    ‏2013-10-11T05:01:24Z  

    Hi Kenhygh,

     

    Thanks for your reply, I tried with response type non-xml, same error.

    mpgw (TES_Inbound_MPG)Parse error

    mpgw (TES_Inbound_MPG)rule (TES_Routing_Policy_response): implied action Parse input as XML failed: illegal character 'C' at offset 0 of 

     

    Back end server responding with response code 500 and error message body in html format in this case.

    Is there any way to parse html when response type will be XML/ Non-XML?

     

    Thanks,

    Sri. 

    Can you try turning off  "Process back end errors" option, so that error rule will be invoked instead of response rule and you will get the actual error code from server.

    Hemchan

  • tallurisri
    tallurisri
    103 Posts

    Re: How to parse html content on response rule.

    ‏2013-10-11T14:28:27Z  
    • Hemchan
    • ‏2013-10-11T05:01:24Z

    Can you try turning off  "Process back end errors" option, so that error rule will be invoked instead of response rule and you will get the actual error code from server.

    Hemchan

    Hi Hemchan,

    Thanks for your reply, I did try  turning off  "Process back end errors" option, I got back side connection failure error from DataPower but my client not expecting this error because actual back end server error was "500 Internal server error".

    But my issue is not with error/status code, with or with out  "Process back end errors" option I was able to see error code. In error case(server can not process request) server trying send error message in html body and DP not parsing html message body and sending parsing error message to client. If I use pass-through response type I am able to see back end server's error message but as per company standards I can't use pass-through.  

    I am searching for solution to process html message content(in error case) with out changing request-response type from XML. Any help would be appreciated. 

    Thanks,

    Sri. 

  • Hemchan
    Hemchan
    21 Posts

    Re: How to parse html content on response rule.

    ‏2013-10-12T03:02:24Z  

    Hi Hemchan,

    Thanks for your reply, I did try  turning off  "Process back end errors" option, I got back side connection failure error from DataPower but my client not expecting this error because actual back end server error was "500 Internal server error".

    But my issue is not with error/status code, with or with out  "Process back end errors" option I was able to see error code. In error case(server can not process request) server trying send error message in html body and DP not parsing html message body and sending parsing error message to client. If I use pass-through response type I am able to see back end server's error message but as per company standards I can't use pass-through.  

    I am searching for solution to process html message content(in error case) with out changing request-response type from XML. Any help would be appreciated. 

    Thanks,

    Sri. 

    "Convert Query Parameters to XML" action from advanced icon as first action in response rule which will convert non-xml to xml and you can use this xml for processing. Am not sure whether this is right solution but you can achieve what you want.

     

  • SriniDp
    SriniDp
    46 Posts

    Re: How to parse html content on response rule.

    ‏2013-10-14T06:01:05Z  
    • Hemchan
    • ‏2013-10-12T03:02:24Z

    "Convert Query Parameters to XML" action from advanced icon as first action in response rule which will convert non-xml to xml and you can use this xml for processing. Am not sure whether this is right solution but you can achieve what you want.

     

    Hi,

     

    Looks like your are trying to parse HTML content in transformation action. If you use transform action with INPUT context as INPUT, datapower is trying to parse non-xml data as XML data, thats how your getting error. TRY to Make NULL as INput context . And what exaclty your looking in HTML content?

  • HermannSW
    HermannSW
    4728 Posts

    Re: How to parse html content on response rule.

    ‏2013-10-14T07:16:06Z  

    Regardless on whether the HTTP response code is 401 (Not Authorized) or 404 (Not found), this happens on the request side of processing.

    Yes, error rule gets called in case you set "Process Backend Errors" to off.

    But from DataPower point of view the error (0x01130006) is just this:
    'HTTP/1.1 500 Error X-Backside-Transport: FAIL FAIL Content-Type: text/xml Connection: close '

    I have found no way to even get the HTTP response code via 'x-dp-response-code', neither by dp:http-response-header()
    nor by dp:http-request-header(),perhaps others can.


    Hermann<myXsltBlog/> <myXsltTweets/> <myCE/>

  • tallurisri
    tallurisri
    103 Posts

    Re: How to parse html content on response rule.

    ‏2013-10-14T20:28:43Z  
    • SriniDp
    • ‏2013-10-14T06:01:05Z

    Hi,

     

    Looks like your are trying to parse HTML content in transformation action. If you use transform action with INPUT context as INPUT, datapower is trying to parse non-xml data as XML data, thats how your getting error. TRY to Make NULL as INput context . And what exaclty your looking in HTML content?

    Hi Srini,

    Thanks for your reply. I am not using transform action response rule contain only match and result actions.

    ####And what exaclty your looking in HTML content?

    Clint want to see whole error message( in html) from server. Any thoughts please. 

    Thanks,

    Sri.

     

  • SriniDp
    SriniDp
    46 Posts

    Re: How to parse html content on response rule.

    ‏2013-10-14T21:04:17Z  

    Hi Srini,

    Thanks for your reply. I am not using transform action response rule contain only match and result actions.

    ####And what exaclty your looking in HTML content?

    Clint want to see whole error message( in html) from server. Any thoughts please. 

    Thanks,

    Sri.

     

    Yes, You can capture non-xml content being the rule as xml type. You can do this by binary transfromation action with ffd in it, by that you can capture the response from the server. Done it many times to capture the non-xml response from webservers.

  • HermannSW
    HermannSW
    4728 Posts

    Re: How to parse html content on response rule.

    ‏2013-10-15T10:25:41Z  
    • SriniDp
    • ‏2013-10-14T21:04:17Z

    Yes, You can capture non-xml content being the rule as xml type. You can do this by binary transfromation action with ffd in it, by that you can capture the response from the server. Done it many times to capture the non-xml response from webservers.

    Hi,

    as I mentioned in my previous posting I did try exactly that and it did not work for me.

    If you were successful in doing what you stated for 401 (Not Authorized) or 404 (Not found) HTTP response errors, can you please attach a small sample service export here?

    I just used a non-existing URL for my 404 testing (http://stamm-wilbrandt.de/notFound).
    For good case testing I did use (http://stamm-wilbrandt.de/ab.xml).


    Hermann<myXsltBlog/> <myXsltTweets/> <myCE/>

  • tallurisri
    tallurisri
    103 Posts

    Re: How to parse html content on response rule.

    ‏2013-10-15T15:49:19Z  
    • HermannSW
    • ‏2013-10-15T10:25:41Z

    Hi,

    as I mentioned in my previous posting I did try exactly that and it did not work for me.

    If you were successful in doing what you stated for 401 (Not Authorized) or 404 (Not found) HTTP response errors, can you please attach a small sample service export here?

    I just used a non-existing URL for my 404 testing (http://stamm-wilbrandt.de/notFound).
    For good case testing I did use (http://stamm-wilbrandt.de/ab.xml).


    Hermann<myXsltBlog/> <myXsltTweets/> <myCE/>

    Hi Hermann,

    I created Service with below configuration options with in MPG, and it is able to pass back end server response codes(500/404/401) to client.

    Request/Response type: XML

    Proxy HTTP Response: ON

    Process Backend Errors: ON/OFF (I made it "ON" to see XML error message body).

    Above configuration working fine for me to pass response codes(500/404/401) and error response body(if it is in XML) from server. 

    Parsing issue is encountering(during response process in DP) when response body is not in XML format , I added below configuration options in MPG to overcome this paring issue:

    Non-XML Processing: ON (in response processing rule).

    Process Backend Errors: ON

    Now my service successfully passing error response code and error response body(either in XML/HTML/plain text) from server to client with out any parsing issues, but I want to know is there any security compromise with above two options "Non-XML Processing: ON" and "Process Backend Errors: ON" ? 

    Find attachment  for sample service export.

    Please share your thoughts.

    Thanks,

    Sri.

     

     

     

     

     

     

    Attachments

  • tallurisri
    tallurisri
    103 Posts

    Re: How to parse html content on response rule.

    ‏2013-10-15T15:51:39Z  
    • SriniDp
    • ‏2013-10-14T21:04:17Z

    Yes, You can capture non-xml content being the rule as xml type. You can do this by binary transfromation action with ffd in it, by that you can capture the response from the server. Done it many times to capture the non-xml response from webservers.

    Hi Srini,

     

    Thanks for your reply, i will try binary transformation action with ffd and let you know the result.

    Thanks,

    Sri.

  • HermannSW
    HermannSW
    4728 Posts

    Re: How to parse html content on response rule.

    ‏2013-10-15T16:55:18Z  

    Hi Hermann,

    I created Service with below configuration options with in MPG, and it is able to pass back end server response codes(500/404/401) to client.

    Request/Response type: XML

    Proxy HTTP Response: ON

    Process Backend Errors: ON/OFF (I made it "ON" to see XML error message body).

    Above configuration working fine for me to pass response codes(500/404/401) and error response body(if it is in XML) from server. 

    Parsing issue is encountering(during response process in DP) when response body is not in XML format , I added below configuration options in MPG to overcome this paring issue:

    Non-XML Processing: ON (in response processing rule).

    Process Backend Errors: ON

    Now my service successfully passing error response code and error response body(either in XML/HTML/plain text) from server to client with out any parsing issues, but I want to know is there any security compromise with above two options "Non-XML Processing: ON" and "Process Backend Errors: ON" ? 

    Find attachment  for sample service export.

    Please share your thoughts.

    Thanks,

    Sri.

     

     

     

     

     

     

    Thanks Sri,

    you made it work, I must have configured something wrong in my previous attempts.

    I did minimize the config, and modified slightly, find new service export attached:

    1. deleted request rule, did set request type to pass-thru
    2. deleted error rule (not needed because you configured "Process Backend Errors" as "on")
    3. replaced Results action in response rule by Transform Binary Action (local:///to-utf8-html.xsl)
    4. did set "static-backend" to "http;//stamm-wilbrandt.de" (because route action is gone with step 1)


    > ... but I want to know is there any security compromise with above two options "Non-XML Processing: ON" and "Process Backend Errors: ON" ?
    >
    not if you do input validation as stylesheet to-utf8-html.xsl does.

    For Non-XML processing the Transform Binary stylesheet needs to know the backend encoding.
    My web server (http://stamm-wilbrandt.de) does return UTF-8 encoded data.

    You can find the stylesheet in attached export, by clicking on "all" link in "files" section of zip2html output, or here.

    zip2html tool shows nicely the minimal config:
    http://stamm-wilbrandt.de/en/forum/Test_MPG-zip2.html
     


    The static backend allows to demonstrate good and bad response, because the URL gets propagated

    This is error response, as you can see you can easily access the response code as well as the HTML body:

    $ curl  http://dp3-l3.boeblingen.de.ibm.com:9332/notFound -sv | tidy -q -xml
    * About to connect() to dp3-l3.boeblingen.de.ibm.com port 9332 (#0)
    *   Trying 9.152.92.61... connected
    * Connected to dp3-l3.boeblingen.de.ibm.com (9.152.92.61) port 9332 (#0)
    > GET /notFound HTTP/1.1
    > User-Agent: curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.14.3.0 zlib/1.2.3 libidn/1.18 libssh2/1.4.2
    > Host: dp3-l3.boeblingen.de.ibm.com:9332
    > Accept: */*
    >
    < HTTP/1.1 404 Not Found
    < X-Backside-Transport: FAIL FAIL
    < Connection: Keep-Alive
    < Transfer-Encoding: chunked
    < Date: Tue, 15 Oct 2013 16:32:26 GMT
    < Server: Apache/2.2.3 (Debian) mod_python/3.2.10 Python/2.4.4 PHP/5.2.0-8+etch16 mod_ssl/2.2.3 OpenSSL/0.9.8c
    < Content-Type: text/html
    < X-Client-IP: 9.152.201.129
    <
    { [data not shown]
    * Connection #0 to host dp3-l3.boeblingen.de.ibm.com left intact
    * Closing connection #0
    <response>
      <rc>404 Not Found</rc>
      <body>&lt;!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN"&gt;
      &lt;html&gt;&lt;head&gt; &lt;title&gt;404 Not Found&lt;/title&gt;
      &lt;/head&gt;&lt;body&gt; &lt;h1&gt;Not Found&lt;/h1&gt;
      &lt;p&gt;The requested URL /notFound was not found on this
      server.&lt;/p&gt; &lt;hr&gt; &lt;address&gt;Apache/2.2.3 (Debian)
      mod_python/3.2.10 Python/2.4.4 PHP/5.2.0-8+etch16 mod_ssl/2.2.3
      OpenSSL/0.9.8c Server at stamm-wilbrandt.de Port
      80&lt;/address&gt; &lt;/body&gt;&lt;/html&gt;</body>
    </response>

    $


    And this is good case, with XML response:

    $ curl  http://dp3-l3.boeblingen.de.ibm.com:9332/ab.xml -sv | tidy -q -xml
    * About to connect() to dp3-l3.boeblingen.de.ibm.com port 9332 (#0)
    *   Trying 9.152.92.61... connected
    * Connected to dp3-l3.boeblingen.de.ibm.com (9.152.92.61) port 9332 (#0)
    > GET /ab.xml HTTP/1.1
    > User-Agent: curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.14.3.0 zlib/1.2.3 libidn/1.18 libssh2/1.4.2
    > Host: dp3-l3.boeblingen.de.ibm.com:9332
    > Accept: */*
    >
    < HTTP/1.1 200 OK
    < X-Backside-Transport: OK OK
    < Connection: Keep-Alive
    < Transfer-Encoding: chunked
    < Date: Tue, 15 Oct 2013 15:50:45 GMT
    < Server: Apache/2.2.3 (Debian) mod_python/3.2.10 Python/2.4.4 PHP/5.2.0-8+etch16 mod_ssl/2.2.3 OpenSSL/0.9.8c
    < Last-Modified: Wed, 19 Dec 2012 19:40:57 GMT
    < ETag: "be15ed-12-cb0fe440"
    < Content-Type: text/html
    < Content-Language: de
    < Age: 2555
    < X-Client-IP: 9.152.201.129
    <
    { [data not shown]
    * Connection #0 to host dp3-l3.boeblingen.de.ibm.com left intact
    * Closing connection #0
    <response>
      <rc>200 OK</rc>
      <body>&lt;a&gt;1&lt;b&gt;2&lt;/b&gt;3&lt;/a&gt;</body>
    </response>

    $


    In case of "200 OK" reponse code you can work further on XML response (if needed) by eg.:

    select="dp:parse($str)/*/*"


    For HTML reponse you cannot use dp:parse() even skipping DOCTYPE.
    The "<hr>" element returned does not have closing tag:

    $ curl  http://stamm-wilbrandt.de/notFound -s
    <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
    <html><head>
    <title>404 Not Found</title>
    </head><body>
    <h1>Not Found</h1>
    <p>The requested URL /notFound was not found on this server.</p>
    <hr>
    <address>Apache/2.2.3 (Debian) mod_python/3.2.10 Python/2.4.4 PHP/5.2.0-8+etch16 mod_ssl/2.2.3 OpenSSL/0.9.8c Server at stamm-wilbrandt.de Port 80</address>
    </body></html>
    $



    Hermann<myXsltBlog/> <myXsltTweets/> <myCE/>

     

    Attachments

    Updated on 2013-10-15T17:27:10Z at 2013-10-15T17:27:10Z by HermannSW
  • tallurisri
    tallurisri
    103 Posts

    Re: How to parse html content on response rule.

    ‏2013-10-15T17:57:10Z  
    • HermannSW
    • ‏2013-10-15T16:55:18Z

    Thanks Sri,

    you made it work, I must have configured something wrong in my previous attempts.

    I did minimize the config, and modified slightly, find new service export attached:

    1. deleted request rule, did set request type to pass-thru
    2. deleted error rule (not needed because you configured "Process Backend Errors" as "on")
    3. replaced Results action in response rule by Transform Binary Action (local:///to-utf8-html.xsl)
    4. did set "static-backend" to "http;//stamm-wilbrandt.de" (because route action is gone with step 1)


    > ... but I want to know is there any security compromise with above two options "Non-XML Processing: ON" and "Process Backend Errors: ON" ?
    >
    not if you do input validation as stylesheet to-utf8-html.xsl does.

    For Non-XML processing the Transform Binary stylesheet needs to know the backend encoding.
    My web server (http://stamm-wilbrandt.de) does return UTF-8 encoded data.

    You can find the stylesheet in attached export, by clicking on "all" link in "files" section of zip2html output, or here.

    zip2html tool shows nicely the minimal config:
    http://stamm-wilbrandt.de/en/forum/Test_MPG-zip2.html
     


    The static backend allows to demonstrate good and bad response, because the URL gets propagated

    This is error response, as you can see you can easily access the response code as well as the HTML body:

    $ curl  http://dp3-l3.boeblingen.de.ibm.com:9332/notFound -sv | tidy -q -xml
    * About to connect() to dp3-l3.boeblingen.de.ibm.com port 9332 (#0)
    *   Trying 9.152.92.61... connected
    * Connected to dp3-l3.boeblingen.de.ibm.com (9.152.92.61) port 9332 (#0)
    > GET /notFound HTTP/1.1
    > User-Agent: curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.14.3.0 zlib/1.2.3 libidn/1.18 libssh2/1.4.2
    > Host: dp3-l3.boeblingen.de.ibm.com:9332
    > Accept: */*
    >
    < HTTP/1.1 404 Not Found
    < X-Backside-Transport: FAIL FAIL
    < Connection: Keep-Alive
    < Transfer-Encoding: chunked
    < Date: Tue, 15 Oct 2013 16:32:26 GMT
    < Server: Apache/2.2.3 (Debian) mod_python/3.2.10 Python/2.4.4 PHP/5.2.0-8+etch16 mod_ssl/2.2.3 OpenSSL/0.9.8c
    < Content-Type: text/html
    < X-Client-IP: 9.152.201.129
    <
    { [data not shown]
    * Connection #0 to host dp3-l3.boeblingen.de.ibm.com left intact
    * Closing connection #0
    <response>
      <rc>404 Not Found</rc>
      <body>&lt;!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN"&gt;
      &lt;html&gt;&lt;head&gt; &lt;title&gt;404 Not Found&lt;/title&gt;
      &lt;/head&gt;&lt;body&gt; &lt;h1&gt;Not Found&lt;/h1&gt;
      &lt;p&gt;The requested URL /notFound was not found on this
      server.&lt;/p&gt; &lt;hr&gt; &lt;address&gt;Apache/2.2.3 (Debian)
      mod_python/3.2.10 Python/2.4.4 PHP/5.2.0-8+etch16 mod_ssl/2.2.3
      OpenSSL/0.9.8c Server at stamm-wilbrandt.de Port
      80&lt;/address&gt; &lt;/body&gt;&lt;/html&gt;</body>
    </response>

    $


    And this is good case, with XML response:

    $ curl  http://dp3-l3.boeblingen.de.ibm.com:9332/ab.xml -sv | tidy -q -xml
    * About to connect() to dp3-l3.boeblingen.de.ibm.com port 9332 (#0)
    *   Trying 9.152.92.61... connected
    * Connected to dp3-l3.boeblingen.de.ibm.com (9.152.92.61) port 9332 (#0)
    > GET /ab.xml HTTP/1.1
    > User-Agent: curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.14.3.0 zlib/1.2.3 libidn/1.18 libssh2/1.4.2
    > Host: dp3-l3.boeblingen.de.ibm.com:9332
    > Accept: */*
    >
    < HTTP/1.1 200 OK
    < X-Backside-Transport: OK OK
    < Connection: Keep-Alive
    < Transfer-Encoding: chunked
    < Date: Tue, 15 Oct 2013 15:50:45 GMT
    < Server: Apache/2.2.3 (Debian) mod_python/3.2.10 Python/2.4.4 PHP/5.2.0-8+etch16 mod_ssl/2.2.3 OpenSSL/0.9.8c
    < Last-Modified: Wed, 19 Dec 2012 19:40:57 GMT
    < ETag: "be15ed-12-cb0fe440"
    < Content-Type: text/html
    < Content-Language: de
    < Age: 2555
    < X-Client-IP: 9.152.201.129
    <
    { [data not shown]
    * Connection #0 to host dp3-l3.boeblingen.de.ibm.com left intact
    * Closing connection #0
    <response>
      <rc>200 OK</rc>
      <body>&lt;a&gt;1&lt;b&gt;2&lt;/b&gt;3&lt;/a&gt;</body>
    </response>

    $


    In case of "200 OK" reponse code you can work further on XML response (if needed) by eg.:

    select="dp:parse($str)/*/*"


    For HTML reponse you cannot use dp:parse() even skipping DOCTYPE.
    The "<hr>" element returned does not have closing tag:

    $ curl  http://stamm-wilbrandt.de/notFound -s
    <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
    <html><head>
    <title>404 Not Found</title>
    </head><body>
    <h1>Not Found</h1>
    <p>The requested URL /notFound was not found on this server.</p>
    <hr>
    <address>Apache/2.2.3 (Debian) mod_python/3.2.10 Python/2.4.4 PHP/5.2.0-8+etch16 mod_ssl/2.2.3 OpenSSL/0.9.8c Server at stamm-wilbrandt.de Port 80</address>
    </body></html>
    $



    Hermann<myXsltBlog/> <myXsltTweets/> <myCE/>

     

    Thanks Hermann,

    I will try my service with Transform Binary Action, as per my understanding this Transform Binary Action will enforce validation to non-xml message content even if I use request/ response type pass-thru. Please correct me if my assumption is wrong.

    I want to keep request and response rules in my service because this is one among the other services in same MPGW. Thanks for your time, I let you know the results with Transform Binary Action.

    Thanks,

    Sri. 

     

     

     

     

  • HermannSW
    HermannSW
    4728 Posts

    Re: How to parse html content on response rule.

    ‏2013-10-16T10:18:36Z  

    Thanks Hermann,

    I will try my service with Transform Binary Action, as per my understanding this Transform Binary Action will enforce validation to non-xml message content even if I use request/ response type pass-thru. Please correct me if my assumption is wrong.

    I want to keep request and response rules in my service because this is one among the other services in same MPGW. Thanks for your time, I let you know the results with Transform Binary Action.

    Thanks,

    Sri. 

     

     

     

     

    Hi,

    I did re-add the request rule (and switched "Process Messages Whose Body Is  Empty" to "on").

    The previously posted service did run the xform action not only for bad response, but for all.

    The new service export attached now does dispatch between "200 OK" and other messages.
    For "200 OK" the response rule gets called conditionally, otherwise the error rule gets called conditionally.

    Important is that the Call Processing Rule action doing that have INPUT as input context and go to OUTPUT.
    This allows both actions (xformbin as well as xform action in normal response rule) to work fine.

    I did add "store:///identity.xsl" xform action to normal response rule just to prove that really XML processing happens,

    And all this works with response type XML because error rule is marked for Non-XML processing.

    It is not possible to match on "x-dp-response-code", so a little trick was needed.
    Stylesheet "local:///resp.xsl" in dispatcher rule xform action (with NULL input context!) stores that value in PIPE context.
    The following conditional action just makes its decison based on PIPE context.

    Here is a complete view of the (now 4) rules, with dispatcher rule highlighted:


    This is the auxiliary stylesheet:

    $ cat resp.xsl
    <xsl:stylesheet version="1.0"
      xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
      xmlns:dp="http://www.datapower.com/extensions"
      extension-element-prefixes="dp"
    >
      <xsl:output omit-xml-declaration="yes"/>
        
      <xsl:template match="/">
        <rc>
          <xsl:value-of select="dp:http-response-header('x-dp-response-code')"/>
        </rc>
      </xsl:template>
     
    </xsl:stylesheet>
    $


    My server does return ab.xml without XML declaration:

    $ curl  http://stamm-wilbrandt.de/ab.xml
    <a>1<b>2</b>3</a>
    $


    The added XML declaration proves that "store:///identity.xsl" was run on normal response rule:

    $ curl  http://dp3-l3.boeblingen.de.ibm.com:9332/ab.xml  ; echo
    <?xml version="1.0" encoding="UTF-8"?>
    <a>1<b>2</b>3</a>
    $

     

    Finally response different to "200 OK" executes the Error rule with the xformbin action as needed:

    $ curl  http://dp3-l3.boeblingen.de.ibm.com:9332/notFound ; echo
    <response><rc>404 Not Found</rc><body>&lt;!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
    &lt;html>&lt;head>
    &lt;title>404 Not Found&lt;/title>
    &lt;/head>&lt;body>
    &lt;h1>Not Found&lt;/h1>
    &lt;p>The requested URL /notFound was not found on this server.&lt;/p>
    &lt;hr>
    &lt;address>Apache/2.2.3 (Debian) mod_python/3.2.10 Python/2.4.4 PHP/5.2.0-8+etch16 mod_ssl/2.2.3 OpenSSL/0.9.8c Server at stamm-wilbrandt.de Port 80&lt;/address>
    &lt;/body>&lt;/html>
    </body></response>
    $


    Nice "x-dp-response-code disspatcher", I was not able to accomplish the same service behavior without it.


    Hermann<myXsltBlog/> <myXsltTweets/> <myCE/>

    Attachments

    Updated on 2013-10-16T10:28:15Z at 2013-10-16T10:28:15Z by HermannSW
  • tallurisri
    tallurisri
    103 Posts

    Re: How to parse html content on response rule.

    ‏2013-10-16T17:16:26Z  
    • HermannSW
    • ‏2013-10-16T10:18:36Z

    Hi,

    I did re-add the request rule (and switched "Process Messages Whose Body Is  Empty" to "on").

    The previously posted service did run the xform action not only for bad response, but for all.

    The new service export attached now does dispatch between "200 OK" and other messages.
    For "200 OK" the response rule gets called conditionally, otherwise the error rule gets called conditionally.

    Important is that the Call Processing Rule action doing that have INPUT as input context and go to OUTPUT.
    This allows both actions (xformbin as well as xform action in normal response rule) to work fine.

    I did add "store:///identity.xsl" xform action to normal response rule just to prove that really XML processing happens,

    And all this works with response type XML because error rule is marked for Non-XML processing.

    It is not possible to match on "x-dp-response-code", so a little trick was needed.
    Stylesheet "local:///resp.xsl" in dispatcher rule xform action (with NULL input context!) stores that value in PIPE context.
    The following conditional action just makes its decison based on PIPE context.

    Here is a complete view of the (now 4) rules, with dispatcher rule highlighted:


    This is the auxiliary stylesheet:

    $ cat resp.xsl
    <xsl:stylesheet version="1.0"
      xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
      xmlns:dp="http://www.datapower.com/extensions"
      extension-element-prefixes="dp"
    >
      <xsl:output omit-xml-declaration="yes"/>
        
      <xsl:template match="/">
        <rc>
          <xsl:value-of select="dp:http-response-header('x-dp-response-code')"/>
        </rc>
      </xsl:template>
     
    </xsl:stylesheet>
    $


    My server does return ab.xml without XML declaration:

    $ curl  http://stamm-wilbrandt.de/ab.xml
    <a>1<b>2</b>3</a>
    $


    The added XML declaration proves that "store:///identity.xsl" was run on normal response rule:

    $ curl  http://dp3-l3.boeblingen.de.ibm.com:9332/ab.xml  ; echo
    <?xml version="1.0" encoding="UTF-8"?>
    <a>1<b>2</b>3</a>
    $

     

    Finally response different to "200 OK" executes the Error rule with the xformbin action as needed:

    $ curl  http://dp3-l3.boeblingen.de.ibm.com:9332/notFound ; echo
    <response><rc>404 Not Found</rc><body>&lt;!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
    &lt;html>&lt;head>
    &lt;title>404 Not Found&lt;/title>
    &lt;/head>&lt;body>
    &lt;h1>Not Found&lt;/h1>
    &lt;p>The requested URL /notFound was not found on this server.&lt;/p>
    &lt;hr>
    &lt;address>Apache/2.2.3 (Debian) mod_python/3.2.10 Python/2.4.4 PHP/5.2.0-8+etch16 mod_ssl/2.2.3 OpenSSL/0.9.8c Server at stamm-wilbrandt.de Port 80&lt;/address>
    &lt;/body>&lt;/html>
    </body></response>
    $


    Nice "x-dp-response-code disspatcher", I was not able to accomplish the same service behavior without it.


    Hermann<myXsltBlog/> <myXsltTweets/> <myCE/>

    Thanks Hermann,

    nice and Interesting stuff you added, I will try it  and let you know results.

    Thanks,

    Sri. 

  • Nik_DPDev
    Nik_DPDev
    6 Posts

    Re: How to parse html content on response rule.

    ‏2013-12-09T22:28:30Z  
    • HermannSW
    • ‏2013-10-14T07:16:06Z

    Regardless on whether the HTTP response code is 401 (Not Authorized) or 404 (Not found), this happens on the request side of processing.

    Yes, error rule gets called in case you set "Process Backend Errors" to off.

    But from DataPower point of view the error (0x01130006) is just this:
    'HTTP/1.1 500 Error X-Backside-Transport: FAIL FAIL Content-Type: text/xml Connection: close '

    I have found no way to even get the HTTP response code via 'x-dp-response-code', neither by dp:http-response-header()
    nor by dp:http-request-header(),perhaps others can.


    Hermann<myXsltBlog/> <myXsltTweets/> <myCE/>

    "Process Backend Errors" as ON and 

    <xsl:variable name="httpResponsePhrase"
                            select="normalize-space(dp:response-header('x-dp-response-code'))" />
                    <xsl:variable name="httpResponseCode"
                            select="substring($httpResponseCodePhrase,1,4 )" />