Topic
  • 3 replies
  • Latest Post - ‏2012-12-24T01:31:59Z by HermannSW
Lakh
Lakh
22 Posts

Pinned topic http post message failing in convert-http node

‏2012-12-19T20:05:43Z |
Hi,

We are getting HTTP post non-xml data in TEST env from external clients. and getting failed with bad MIME format error.

Here is the header content-data

Content-Type multipart/form-data; boundary=###]boundary]###

I tested same with Jmeter in DEV environment and getting below content-type and worked fine.

Content-Type multipart/form-data; boundary=---------------------------7d159c1302d0y0

ASCII data attached.

Any suggestions how we can handle boundary value with ### or ] ??
Updated on 2012-12-24T01:31:59Z at 2012-12-24T01:31:59Z by HermannSW
  • HermannSW
    HermannSW
    4657 Posts

    Re: http post message failing in convert-http node

    ‏2012-12-20T09:10:49Z  
    Form MIME spec http://www.faqs.org/rfcs/rfc1521.html:
    boundary := 0*69<bchars> bcharsnospace
     
       bcharsnospace :=    DIGIT / ALPHA / "'" / "(" / ")" / "+"  / "_"
                      / "," / "-" / "." / "/" / ":" / "=" / "?"
    


    Neither "#" nor "]" are allowed boundary characters.

    The best you can do is making your clients send spec-conform MIME messages.

    If that is not possible, you may do Non-XML preprocessing of the raw input data to replace "#" and "]" along the lines of swaform tool:
    https://www.ibm.com/developerworks/forums/thread.jspa?messageID=14451965&#14451965

     
    Hermann <myXsltBlog/> <myXsltTweets/>
    Updated on 2014-03-25T02:44:28Z at 2014-03-25T02:44:28Z by iron-man
  • Lakh
    Lakh
    22 Posts

    Re: http post message failing in convert-http node

    ‏2012-12-21T17:08:56Z  
    • HermannSW
    • ‏2012-12-20T09:10:49Z
    Form MIME spec http://www.faqs.org/rfcs/rfc1521.html:
    <pre class="java dw" data-editor-lang="java" data-pbcklang="java" dir="ltr">boundary := 0*69<bchars> bcharsnospace bcharsnospace := DIGIT / ALPHA / "'" / "(" / ")" / "+" / "_" / "," / "-" / "." / "/" / ":" / "=" / "?" </pre>

    Neither "#" nor "]" are allowed boundary characters.

    The best you can do is making your clients send spec-conform MIME messages.

    If that is not possible, you may do Non-XML preprocessing of the raw input data to replace "#" and "]" along the lines of swaform tool:
    https://www.ibm.com/developerworks/forums/thread.jspa?messageID=14451965&#14451965

     
    Hermann <myXsltBlog/> <myXsltTweets/>
    Hi HermannSW,

    Thank you for your response.

    I would like to change on Datapower side to replace Boundary value ###]boundary]### to random string. When i use transformation Action after match node it is failing with illegal characters error.

    My understand is it has to be http-convert action after match node for HTTP POST requests. Please confirm? or is there way to use some xsl code in transformation action after match node to work around on these special characters.
  • HermannSW
    HermannSW
    4657 Posts

    Re: http post message failing in convert-http node

    ‏2012-12-24T01:31:59Z  
    • Lakh
    • ‏2012-12-21T17:08:56Z
    Hi HermannSW,

    Thank you for your response.

    I would like to change on Datapower side to replace Boundary value ###]boundary]### to random string. When i use transformation Action after match node it is failing with illegal characters error.

    My understand is it has to be http-convert action after match node for HTTP POST requests. Please confirm? or is there way to use some xsl code in transformation action after match node to work around on these special characters.
    Hi Sub,

    > I would like to change on Datapower side to replace Boundary value ###]boundary]### to random string. ...
    > or is there way to use some xsl code in transformation action after match node to work around on these special characters.
    >
    you should really get your clients send spec conforming request.

    In the meantime, here is a little christmas present for you.

    Here is a small HTTP Form sample file with spec conform boundary:
    
    $ cat tst.disp ------------------------------7f2d33a80bbd Content-Disposition: form-data; name=
    "data"   test ------------------------------7f2d33a80bbd Content-Disposition: form-data; name=
    "data2"   1234 ------------------------------7f2d33a80bbd-- $
    


    As you can see convert-http action just does its job here:
    
    $ curl --data-binary @tst.disp -H 
    "Content-Type: multipart/form-data; boundary=----------------------------7f2d33a80bbd" http:
    //firestar:2054 ; echo <?xml version=
    "1.0" encoding=
    "UTF-8"?> <request><url>/</url><base-url>/</base-url><args src=
    "url"/><args src=
    "body"><arg name=
    "data">test</arg><arg name=
    "data2">1234</arg></args></request> $
    


    Now this is the small HTTP Form sample with your bad boundary:
    
    $ cat tst2.disp --###]boundary]### Content-Disposition: form-data; name=
    "data"   test --###]boundary]### Content-Disposition: form-data; name=
    "data2"   1234 --###]boundary]###-- $
    


    You have to send this against another endpoint. The Non-XML service at 2055 has just a
    Transform Binary action with stylesheet boundary-fix.xsl (also attached) and goes against
    127.0.0.1:2054 backend. Here you can see that it really works with bad boundary:
    
    $ curl --data-binary @tst2.disp -H 
    "Content-Type: multipart/form-data; boundary=###]boundary]###" http:
    //firestar:2055 ; echo <?xml version=
    "1.0" encoding=
    "UTF-8"?> <request><url>/</url><base-url>/</base-url><args src=
    "url"/><args src=
    "body"><arg name=
    "data">test</arg><arg name=
    "data2">1234</arg></args></request> $
    


    Here is stylesheet boundary-fix.xsl for completeness, important is the setting of new boundary in Content-Type:
    
    $ cat boundary-fix.xsl <xsl:stylesheet version=
    "1.0" xmlns:xsl=
    "http://www.w3.org/1999/XSL/Transform" xmlns:regexp=
    "http://exslt.org/regular-expressions" xmlns:dp=
    "http://www.datapower.com/extensions" extension-element-prefixes=
    "dp" > <dp:input-mapping  href=
    "store:///pkcs7-convert-input.ffd" type=
    "ffd"/>   <xsl:output omit-xml-declaration=
    "yes" /> <!-- get us a unique MIME boundary --> <xsl:variable name=
    "to"   select=
    "dp:generate-uuid()"/> <xsl:variable name=
    "from" select=
    "'###]boundary]###'"/>     <xsl:template match=
    "/"> <xsl:variable name=
    "input64" select=
    "dp:binary-encode(/object/message/node())" /> <!-- 
    
    this does UTF-8 validation and decoding --> <xsl:variable name=
    "str" select=
    "dp:decode($input64, 'base-64')"/>   <xsl:value-of select=
    "regexp:replace($str, $from, 'g', $to)"/>   <dp:set-http-request-header name=
    "'Content-Type'" value=
    "concat('multipart/form-data; boundary=',$to)" /> </xsl:template> </xsl:stylesheet> $
    


    This is perhaps a too simple fix, but a good starter.
    It will not work if a MIME section would contain 0x00 bytes.
    Also, Non-UTF-8 data in a MIME section may confuse the regexp:replace().
    There are solutions for dealing with other encodings, but only one at a time.
    If more than one encoding is needed for different MIME sections, then doing
    the regexp:replace() on hexadecimally encoded string will do, as in swaform tool posting:
    https://www.ibm.com/developerworks/forums/thread.jspa?messageID=14451965#14451965

     
    Hermann<myXsltBlog/> <myXsltTweets/>