Topic
39 replies Latest Post - ‏2013-07-11T10:59:08Z by HermannSW
HermannSW
HermannSW
2818 Posts
ACCEPTED ANSWER

Pinned topic swaform -- using HTTP forms as generator for "Soap With Attachments"

‏2010-03-31T20:28:34Z |
Hello,

I had the idea to use any HTML form as generator of SWA requests (Soap With Attachments) against a DataPower box.

There were two technical issues to be solved. Browser sends
A) Content-Type "multipart/form-data"
B) 'Content-Disposition: form-data; name="photo"; ...'

while DataPower SWA handling expects
A) Content-Type "multipart/related"
B) 'Content-Id: <photo>'

Today I learned from a colleague how to do the rewrite of the Content-Type (local:///form-data.xsl) in the Protocol Headers (A).

Then I wrote a little stylesheet (local:///Disposition.xsl) replacing 'Content-Disposition' as requested by (B) for all fields of the HTML form.

Additionally there is a little HTTP Service on DataPower with a demonstration HTTP form as default page.

These are the services of swaform domain:
30080: HTTP Service (swaformserver)
30081: XML FW (swaform)
30082: XML FW (swaloopback)

How to install+test:
a) Import attached backup on your box.
b) switch to swaform domain
c) in FileManager edit file local:///upload3.html
d) replace "datapower1.boeblingen.de.ibm.com" by your boxes hostname or IP address and save
e) open "http://yourbox:30080" in your Webbrowser
f) select a file to upload as "photo"
g) enter anything into email address field
h) click submit
i) save the multipart/related response
j) open the Probe (intentionally active in backup) of XML FW swaloopback
k) inspect Content-Type under Headers
l) inspect the Attachments
This little tool (swaform) is
  • available on any DataPower box
  • you do not need a program (eg. curl) to send the requests
  • you do not need to deal with SWA boundaries
  • you do not need a seperate Webserver hosting the HTML form
  • can be used as "multipart/form-data" to SWA converter (the file you saved (i) is a SWA file).

The backup was created under 3.8.0.3, but I successfully imported it under 3.7.3.9.
Have fun with sawform!



For easy lookup:

upload3.html (the first input of type "hidden" provides the SOAP needed):

<!DOCTYPE html PUBLIC 
"-//W3C//DTD XHTML 1.0 Strict//EN" 
"DTD/xhtml1-strict.dtd"> <html xmlns=
"http://www.w3.org/1999/xhtml" xml:lang=
"en" lang=
"en"> <head> <meta http-equiv=
"Content-Type" content=
"text/html; charset=utf-8" /> <title>File Upload</title> </head> <body> <form action=
"http://dp1-l3.boeblingen.de.ibm.com:30081" method=
"post" enctype=
"multipart/form-data"> <input name=
"soap" type=
"hidden" value=
"<env:Envelope xmlns:env='http://schemas.xmlsoap.org/soap/envelope/'><env:Body/></env:Envelope>" > <p>Photo to Upload: <input type=
"file" name=
"photo" /></p> <p>Your Email Address: <input type=
"text" name=
"email_address" /></p> <p><input type=
"submit" name=
"Submit" value=
"Submit Form" /></p> </form> </body> </html>


form-data.xsl:

<xsl:stylesheet version=
"1.0" xmlns:xsl=
"http://www.w3.org/1999/XSL/Transform" xmlns:dp=
"http://www.datapower.com/extensions" xmlns:regexp=
"http://exslt.org/regular-expressions" extension-element-prefixes=
"dp" > <xsl:template match=
"/"> <xsl:variable name=
"original" select=
"dp:http-request-header('Content-Type')"/> <xsl:variable name=
"modified" select=
"regexp:replace($original,'form-data','','related')"/> <dp:set-http-request-header name=
"'Content-Type'" value=
"$modified"/> <dp:freeze-headers/> </xsl:template> </xsl:stylesheet>


Disposition.xsl:

<xsl:stylesheet version=
"1.0" xmlns:xsl=
"http://www.w3.org/1999/XSL/Transform" xmlns:dp=
"http://www.datapower.com/extensions" xmlns:regexp=
"http://exslt.org/regular-expressions" extension-element-prefixes=
"dp" > <xsl:output method=
"text" /> <dp:input-mapping  href=
"hexBinary.ffd" type=
"ffd"/> <dp:output-mapping href=
"hexBinary.ffd" type=
"ffd"/>     <!-- ^Content-Disposition:[^
"]"([^
"]*).*$ --> <xsl:variable name=
"from" select=
"'0a436f6e74656e742d446973706f736974696f6e3a(([013-9a-f][0-9a-f]|2[013-9a-f])*)22(([013-9a-f][0-9a-f]|2[013-9a-f])*)(([1-9a-f][0-9a-f]|0[0-9b-f])*)'" /> <!-- 0a 

for ^ and $ (([1-9a-f][0-9a-f]|0[0-9b-f])*) 

for 
"any byte until 0a" -->     <!-- Content-Id: <\1> --> <xsl:variable name=
"to" select=
"'0a436f6e74656e742d49643a203c$33e'" /> <!-- 0a 

for ^ $3 instead \1 because of the additional parenthesis pairs -->     <xsl:template match=
"Conversion/hexstr"> <Conversion> <hexstr> <xsl:copy-of select=
"regexp:replace(.,$from,'g',$to)" /> </hexstr> </Conversion> </xsl:template> </xsl:stylesheet>


hexBinary.ffd (for completeness):

<?xml version=
"1.0" encoding=
"UTF-8" ?> <File version=
"2.1" name=
"Conversion"> <Field name=
"hexstr" type=
"hexBinary" /> </File>
Updated on 2013-03-11T19:17:12Z at 2013-03-11T19:17:12Z by SystemAdmin
  • HermannSW
    HermannSW
    2818 Posts
    ACCEPTED ANSWER

    Re: swaform -- using HTTP forms as generator for "Soap With Attachments"

    ‏2010-04-07T16:31:50Z  in response to HermannSW
    I was asked by a colleague whether swaform tool could be modified to process arbitrary HTML form data.
    The reason was that the hidden field containing a dummy SOAP message as shown above in upload3.html is not available if the HTML forms cannot be changed.

    I changed Disposition.xsl to just prepend a dummy SOAP header and removed the hidden HTML input field from upload3.html sample, that's all.

    Now any HTML form data can be processed by swaform tool (the attached backup of swaform domain contains all changes).

    Changed section of Dispostion.xsl:
    
    ... <xsl:template match=
    "Conversion/hexstr"> <Conversion> <hexstr> <xsl:variable name=
    "hexmsg" select=
    "regexp:replace(.,$from,'g',$to)" />   <!-- boundary is anything until first linefeed (0a) --> <xsl:variable name=
    "boundary" select=
    "regexp:match($hexmsg,'^(([1-9a-f][0-9a-f]|0[0-9b-f])*)(([1-9a-f][0-9a-f]|0[0-9b-f])*)','')"/>   <!-- prepend boundary --> <xsl:value-of select=
    "$boundary"/>0a   <!-- prepend hex version of 
    
    this (as dummy SOAP message): Content-Type: text/xml; charset=UTF-8   <?xml version=
    "1.0" encoding=
    "UTF-8"?> <env:Envelope xmlns:env=
    "http://schemas.xmlsoap.org/soap/envelope/"><env:Body/></env:Envelope> --> 436f6e74656e742d547970653a20746578742f786d6c3b20636861727365743d 5554462d380a0a3c3f786d6c2076657273696f6e3d22312e302220656e636f64 696e673d225554462d38223f3e0a3c656e763a456e76656c6f706520786d6c6e 733a656e763d22687474703a2f2f736368656d61732e786d6c736f61702e6f72 672f736f61702f656e76656c6f70652f223e3c656e763a426f64792f3e3c2f65 6e763a456e76656c6f70653e0a   <!-- append converted message --> <xsl:value-of select=
    "$hexmsg"/> </hexstr> </Conversion> </xsl:template>   </xsl:stylesheet>
    
    • HermannSW
      HermannSW
      2818 Posts
      ACCEPTED ANSWER

      Re: swaform -- using HTTP forms as generator for "Soap With Attachments"

      ‏2010-04-07T18:42:11Z  in response to HermannSW
      Another colleague asked on how to access the attachments.

      For uploading a .gif file this can be seen in Attachment tab in Probe:
      
      uri                       content-type    content cid:photo               image/gif       (show binary) cid:email_address                 (show binary) cid:Submit                                (show binary)
      

      The uploaded file gets content-type set (depending on the data uploaded).
      For the other attachments the content-type is not set (the text entered into the email address field, and the string "Submit Form" from upload3.html).

      For accessing the attachments with Fetch or Results action see eg. chapter "Attachment protocol" in "XSL Accelerator Developers Guide":
      http://www-01.ibm.com/software/integration/datapower/library/documentation/v3.8.0/XS40.html

      Accessing the attachments from within a stylesheet is done by dp:url-open extension.
      It allows for conversion of the (binary) data of the attachment into hexbin or base64.
      If knowing that the attachment data is text as in the email_address attachment from above this can be accessed too.
      Find below the stylesheet and the output it produces.

      Stylesheet:
      
      <xsl:stylesheet version=
      "1.0" xmlns:xsl=
      "http://www.w3.org/1999/XSL/Transform" xmlns:dp=
      "http://www.datapower.com/extensions" extension-element-prefixes=
      "dp" > <xsl:output method=
      "xml"/>   <xsl:template match=
      "/"> <attachments> <photo><dp:url-open target=
      "cid:photo?Encode=hexbin" /></photo> <xsl:variable name=
      "email"> <dp:url-open target=
      "cid:email_address?Encode=base64" /> </xsl:variable> <xsl:variable name=
      "raw" select=
      "dp:binary-decode($email)" /> <email_address><xsl:value-of select=
      "$raw"/></email_address>   <Submit><dp:url-open target=
      "cid:Submit?Encode=base64" /></Submit> </attachments> </xsl:template> </xsl:stylesheet>
      


      Output (formatted with "tidy -xml"):
      
      <attachments> <photo> <hexbin> 4749463839615d007800e7ff0007010004013200004708006a00017b0316213d00002c1c0d1d 00fc302123072588002e7a05365500386e2b31404433037b1d135423625c3501463d2625476b0b4d ... 622402c17d0620002c0008afe00ee8203612b000cbc32e875410d1e0002ac000134002719037aadb d6b53213956a14959a0db39c34134dd4a0bd10c981d9a37ddaa71d10003b</hexbin> </photo> <email_address>test123</email_address> <Submit> <base64>U3VibWl0IEZvcm0=</base64> </Submit> </attachments>
      


      
      $ base64 -d U3VibWl0IEZvcm0= Submit Form $ tail -4 upload3.html <p><input type=
      "submit" name=
      "Submit" value=
      "Submit Form" /></p> </form> </body> </html> $
      
  • HermannSW
    HermannSW
    2818 Posts
    ACCEPTED ANSWER

    Re: swaform -- using HTTP forms as generator for "Soap With Attachments"

    ‏2010-12-17T17:44:07Z  in response to HermannSW
    Somebody wanted to display an uploaded 'gzip'ed file's content:

    This is the testfile used:
    $ echo "This is a gzip test text" | gzip > test-text.gz
    $
    


    This is how to 'gunzip' the file and how to access it -- "gunzip = dp:inflate(..., 'gzip')":
    <xsl:stylesheet version="1.0" 
      xmlns:xsl="http://www.w3.org/1999/XSL/Transform" 
      xmlns:dp="http://www.datapower.com/extensions"
      extension-element-prefixes="dp"
    >
      <xsl:template match="/">
        <xsl:variable name="rawFile">
            <dp:url-open target="cid:file?Encode=base64"/>
        </xsl:variable>
        <extracted>
          <raw-base64>
            <xsl:attribute name="value">
              <xsl:value-of select="$rawFile"/>
            </xsl:attribute>
          </raw-base64>
          <inflated-gzip>
            <xsl:attribute name="value">
              <xsl:value-of select="dp:inflate($rawFile, 'gzip')"/>
            </xsl:attribute>
          </inflated-gzip>
        </extracted>
      </xsl:template>
    </xsl:stylesheet>
    


    Find attached screenshots of the HTTP form before clicking on submit and the result produced.

    Hermann.
    Updated on 2014-03-25T03:36:31Z at 2014-03-25T03:36:31Z by iron-man
  • Stolbovsky
    Stolbovsky
    22 Posts
    ACCEPTED ANSWER

    Re: swaform -- using HTTP forms as generator for "Soap With Attachments"

    ‏2011-01-19T12:24:07Z  in response to HermannSW
    Hi, Hermann!
    I tried swaform2 to learn file posting opportunities of datapower.
    I've uploaded swaform2 into new domain and tried to upload file with upload3.html form. Trying to find posted file in the tab "Attachments" inside probe I was very disappointed because there was no files at all.
    You wrote in your post that posted files can be seen in the probe tab "Attachments". But can you explain me what I'm doing wrong and how can I see posted files?
    I would appreciate your response.

    Best regards, Dmitry
  • samanderson
    samanderson
    172 Posts
    ACCEPTED ANSWER

    Re: swaform -- using HTTP forms as generator for "Soap With Attachments"

    ‏2011-01-19T15:09:18Z  in response to HermannSW
    Hi Dmirty,

    I have visually verified. No issues with the Post.

    -Sam

    ureachdatapower@gmail.com
  • HermannSW
    HermannSW
    2818 Posts
    ACCEPTED ANSWER

    Re: swaform -- using HTTP forms as generator for "Soap With Attachments"

    ‏2011-01-19T17:00:00Z  in response to HermannSW
    Hi Dmitrie,

    I downloaded freshly "backup2-swaform.zip" from this thread.
    Then I installed the backup on my applience.
    Then I updated file local:///upload3.html as stated in steps c) and d) of my description.
    Then I opened http://mybox:30080 as in step e) and followed the other steps.

    Have you turned on Probe on XML FW swaloopback?
    See attached a screenshot of Probe context view I got, with the three promised attachments in Attachment tab ...

    Hermann.
    • Stolbovsky
      Stolbovsky
      22 Posts
      ACCEPTED ANSWER

      Re: swaform -- using HTTP forms as generator for "Soap With Attachments"

      ‏2011-01-25T14:02:12Z  in response to HermannSW
      Hi, Hermann!
      I am very thankful to you for your reply!
      I was not sufficiently attentive to see probe in swaloopback XML Firewall service. It works.
      Now I'm trying to resolve the task of transferring attached files to backend server through MPG service.
      May be you can tell why Convert Query Params to XML Action causes error while parsing POST HTTP message with multipart/form-data header?

      Thank you!
      • HermannSW
        HermannSW
        2818 Posts
        ACCEPTED ANSWER

        Re: swaform -- using HTTP forms as generator for "Soap With Attachments"

        ‏2011-01-25T14:34:54Z  in response to Stolbovsky
        Hi Dmitry,

        > ...
        > May be you can tell why Convert Query Params to XML Action causes error while
        > parsing POST HTTP message with multipart/form-data header?

        It is not generating an error "normally", see eg. this slide:
        http://www-01.ibm.com/support/docview.wss?uid=swg27019118&aid=1#page=15

        convert-http action is designed to convert HTTP Form data to XML.
        But it does not support file upload fields.
        You can select it in WebGUI as "Convert Query Params to XML" action under "Advanced actions", see attached screenshot.

        Processing of HTTP form data with File upload fields (see screenshot from previous attachment) can be done by swaform tool of this thread.

        Hermann.
  • Liv2luv
    Liv2luv
    573 Posts
    ACCEPTED ANSWER

    Re: swaform -- using HTTP forms as generator for "Soap With Attachments"

    ‏2011-01-25T21:08:41Z  in response to HermannSW
    Hi Hermann,

    Can you point me to some references to understand how you have determined the FROM and TO values in disposition.xsl when changing the headers from Content-Disposition: to Content-Id:

    Thanks.
    • HermannSW
      HermannSW
      2818 Posts
      ACCEPTED ANSWER

      Re: swaform -- using HTTP forms as generator for "Soap With Attachments"

      ‏2011-01-25T22:35:44Z  in response to Liv2luv
      Hi Suresh,
      >
      > Can you point me to some references to understand how you have determined the FROM and TO
      > values in disposition.xsl when changing the headers from Content-Disposition: to Content-Id:
      >

      While the description of "Content-ID" is good
      http://en.wikipedia.org/wiki/Content-type#Content-ID.5B3.5D

      the description of "Content-Disposition" was not really helpful:
      http://en.wikipedia.org/wiki/Content-type#Content-Disposition
      So the easiest way to find out on Content-Disposition can be done with DataPower:
      • setup an HTTP Service on DP hosting attached HTML page with file upload HTTP Form
      • setup a second HTTP Service on DP in "echo mode"
      • point HTTP Form post to second service
      • open the HTML page in browser
      • select a .zip file
      • click submit
      • because of "echo mode" the Form Data posted is returned to the browser, save it

      Now inspecting the saved file shows the Details on "Content-Disposition":
      
      $ od -Ax -tcx1 jihKYcgK.part | head -20 000000   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   - 2d 2d 2d 2d 2d 2d 2d 2d 2d 2d 2d 2d 2d 2d 2d 2d 000010   -   -   -   -   -   -   -   -   -   -   -   -   -   4   6   9 2d 2d 2d 2d 2d 2d 2d 2d 2d 2d 2d 2d 2d 34 36 39 000020   7   9   6   0   4   7   3   8   9   9   7   5   4   7   7   1 37 39 36 30 34 37 33 38 39 39 37 35 34 37 37 31 000030   3   3   0   8   6   8   5 0  \r  \n   C   o   n   t   e   n 33 33 30 38 36 38 35 30 0d 0a 43 6f 6e 74 65 6e 000040   t   -   D   i   s   p   o   s   i   t   i   o   n   :       f 74 2d 44 69 73 70 6f 73 69 74 69 6f 6e 3a 20 66 000050   o   r   m   -   d   a   t   a   ;       n   a   m   e   =   
      " 6f 72 6d 2d 64 61 74 61 3b 20 6e 61 6d 65 3d 22 000060   b   a   c   k   u   p   .   z   i   p   
      "   ;       f   i   l 62 61 63 6b 75 70 2e 7a 69 70 22 3b 20 66 69 6c 000070   e   n   a   m   e   =   
      "   e   x   p   o   r   t   -   b   i 65 6e 61 6d 65 3d 22 65 78 70 6f 72 74 2d 62 69 000080   n   l   o   g   .   z   i   p   
      "  \r  \n   C   o   n   t   e 6e 6c 6f 67 2e 7a 69 70 22 0d 0a 43 6f 6e 74 65 000090   n   t   -   T   y   p   e   :       a   p   p   l   i   c   a 6e 74 2d 54 79 70 65 3a 20 61 70 70 6c 69 63 61 $
      


      The comments in Disposition.xsl state what regexp terms are encoded by the hexadecimal strings.
      Just take the filename between the two quote characters following Content-Disposition
      and use it between "<>" characters of Content-ID.
      I was really surprised that this conversion is so easy ...
      
      ... <!-- ^Content-Disposition:[^
      "]*"([^
      "]*).*$ --> <xsl:variable name=
      "from" select=
      "'0a436f6e74656e742d446973706f736974696f6e3a(([013-9a-f][0-9a-f]|2[013-9a-f])*)22(([013-9a-f][0-9a-f]|2[013-9a-f])*)(([1-9a-f][0-9a-f]|0[0-9b-f])*)'" /> <!-- 0a 
      
      for ^ and $ (([1-9a-f][0-9a-f]|0[0-9b-f])*) 
      
      for 
      "any byte until 0a" -->     <!-- Content-Id: <\1> --> <xsl:variable name=
      "to" select=
      "'0a436f6e74656e742d49643a203c$33e'" /> <!-- 0a 
      
      for ^ $3 instead \1 because of the additional parenthesis pairs --> ...
      


      Hermann.
      • SystemAdmin
        SystemAdmin
        6772 Posts
        ACCEPTED ANSWER

        Re: swaform -- using HTTP forms as generator for "Soap With Attachments"

        ‏2011-08-04T20:31:49Z  in response to HermannSW
        Hi Herman,
        Suppose the request from client coming to datapower is already swa, how to determine the number of attachments, strip these attachment and at the back end send JMS over MQ.
        Pls send the link if these topics are already discussed. Thanks for ur time..
        Regards
        • HermannSW
          HermannSW
          2818 Posts
          ACCEPTED ANSWER

          Re: swaform -- using HTTP forms as generator for "Soap With Attachments"

          ‏2011-08-15T21:09:49Z  in response to SystemAdmin
          Hi,

          > Suppose the request from client coming to datapower is already swa,
          > how to determine the number of attachments,
          >
          see stylesheet "attman.xsl" on how to access "var://local/attachment-manifest":
          http://www.ibm.com/developerworks/forums/thread.jspa?threadID=387511&tstart=0#14672211

          This is the XPath expression for counting the attachments:
          
          $ coproc2swa cid:attman.xsl ~/bin/coproc2swa http:
          //127.0.0.1:2223 -s  | \ >  xpath++ 
          "count(/*/manifest/attachments/attachment)" - 7 $ $ coproc2swa cid:attman.xsl ~/bin/coproc2swa http:
          //127.0.0.1:2223 -s  | \ >  xpath++ 
          "/*/manifest/attachments/attachment" -   ------------------------------------------------------------------------------- <attachment><uri>cid:att1</uri><size>22</size><header><name>Content-Id</name><value><att1></value></header><header><name>Content-Type</name><value>text/plain</value></header></attachment> ------------------------------------------------------------------------------- <attachment><uri>cid:attman.xsl</uri><size>1245</size><header><name>Content-Id</name><value><attman.xsl></value></header><header><name>Content-Type</name><value>text/xml</value></header></attachment> ------------------------------------------------------------------------------- <attachment><uri>cid:att3</uri><size>22</size><header><name>Content-Type</name><value>text/plain</value></header><header><name>Content-Id</name><value><att3></value></header></attachment> ------------------------------------------------------------------------------- <attachment><uri>cid:coproc2swa.pgm</uri><size>2984</size><header><name>Content-Type</name><value>image/x-portable-greymap</value></header><header><name>Content-Transfer-Encoding</name><value>8bit</value></header><header><name>Content-ID</name><value><coproc2swa.pgm></value></header></attachment> ------------------------------------------------------------------------------- <attachment><uri>cid:pgm1st.xsl</uri><size>982</size><header><name>Content-Type</name><value>text/xml</value></header><header><name>Content-Id</name><value><pgm1st.xsl></value></header></attachment> ------------------------------------------------------------------------------- <attachment><uri>cid:xml1st.xsl</uri><size>806</size><header><name>Content-Type</name><value>text/xml</value></header><header><name>Content-Id</name><value><xml1st.xsl></value></header></attachment> ------------------------------------------------------------------------------- <attachment><uri>cid:zipit.xsl</uri><size>3049</size><header><name>Content-Type</name><value>text/xml</value></header><header><name>Content-Id</name><value><zipit.xsl></value></header></attachment> $
          


          > strip these attachment ...
          use <dp:strip-attachments/>
          http://publib.boulder.ibm.com/infocenter/wsdatap/v3r8m1/index.jsp?topic=%2Fxa35%2Fextensionfunctions32.htm&resultof=%22dp%3astrip-attachments%22

           
          Hermann<myXsltBlog/>
      • HeJingang
        HeJingang
        11 Posts
        ACCEPTED ANSWER

        Re: swaform -- using HTTP forms as generator for "Soap With Attachments"

        ‏2013-07-11T08:43:09Z  in response to HermannSW

        Hi Hermann,

        Thanks for your sample code.

        But the Regexp really make me confused.

        We have some additional requirement:

        1. we may have multiple files uploaded with the same "name"

        2. we need to get the filename of each uploaded file.

        How to write the Regexp?

        Thanks very much.

  • SystemAdmin
    SystemAdmin
    6772 Posts
    ACCEPTED ANSWER

    Re: swaform -- using HTTP forms as generator for "Soap With Attachments"

    ‏2012-10-15T04:36:09Z  in response to HermannSW
    Hello HermannSW:

    I downloaded swaform tool to test to understand how I can transform HTTP MIME multipart message to SOAP message. However, I am running into problems with message illegal character '-' at line 2 . All I did was modified the upload3.html with more fields and attached text file in upload field in upload3.html page.

    Content of upload3.html and HTTP MIME echo output are below

    1) Upload3.html

    <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "DTD/xhtml1-strict.dtd">
    <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
    <head>
    <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
    <title>File Upload</title>
    </head>
    <body>
    <form action="http://hets.cms:30080" method="post" enctype="multipart/form-data">

    Upload <input type="file" name="Payload" />

    PayloadType<input type="text" name="PayloadType" value="X12_270_004010X092A1/></p>
    <p>ProcessingMode<input type="text" name="ProcessingMode" value="RealTime"/>

    PayloadID<input type="text" name="PayloadID" value="e51d4fae-7dec-11d0-a765-00a0c91e6da6"/>

    TimeStamp<input type="text" name="TimeStamp" value="2012-08-30T10:20:34Z"/>

    UserName<input type="text" name="UserName" value="hospa"/>

    Password<input type="text" name="Password" value="8y6dt3dd2"/>

    SenderID<input type="text" name="SenderID" value="HospitalA"/>

    ReceiverID<input type="text" name="ReceiverID" value="PayerB"/>

    CORERuleVersion<input type="text" name="CORERuleVersion" value="2.2.0"/>

    <input type="submit" name="Submit" value="Submit Form" />

    </form>
    </body>
    </html>
    2) HTTP MIME echo output

    -----------------------------7dc247f1a01f6
    Content-Disposition: form-data; name="Upload"; filename="C:\test.txt"
    Content-Type: text/plain

    Hello Service
    I am here
    -----------------------------7dc247f1a01f6
    Content-Disposition: form-data; name="PayloadType"

    X12_270_004010X092A1
    -----------------------------7dc247f1a01f6
    Content-Disposition: form-data; name="ProcessingMode"

    RealTime
    -----------------------------7dc247f1a01f6
    Content-Disposition: form-data; name="PayloadID"

    e51d4fae-7dec-11d0-a765-00a0c91e6da6
    -----------------------------7dc247f1a01f6
    Content-Disposition: form-data; name="TimeStamp"

    2012-08-30T10:20:34Z
    -----------------------------7dc247f1a01f6
    Content-Disposition: form-data; name="UserName"

    hospa
    -----------------------------7dc247f1a01f6
    Content-Disposition: form-data; name="Password"

    8y6dt3dd2
    -----------------------------7dc247f1a01f6
    Content-Disposition: form-data; name="SenderID"

    HospitalA
    -----------------------------7dc247f1a01f6
    Content-Disposition: form-data; name="ReceiverID"

    PayerB
    -----------------------------7dc247f1a01f6
    Content-Disposition: form-data; name="CORERuleVersion"

    2.2.0
    -----------------------------7dc247f1a01f6
    Content-Disposition: form-data; name="Submit"

    Submit Form
    -----------------------------7dc247f1a01f6--
    My goal is to transform above MIME message to SOAP XML message like below.

    <soapenv:Envelope xmlns:soapenv="http://www.w3.org/2003/05/soap-envelope">
    <soapenv:Header />
    <soapenv:Body>
    <ns1:COREEnvelopeRealTimeRequest xmlns:ns1="http://www.caqh.org/SOAP/WSDL/CORERule2.2.0.xsd">
    <PayloadType>X12_270_004010X092A1</PayloadType>
    <ProcessingMode>RealTime</ProcessingMode>
    <PayloadID>e51d4fae-7dec-11d0-a765-00a0c91e6da6</PayloadID>
    <TimeStamp>2012-08-30T10:20:34Z</TimeStamp>
    <SenderID>HospitalA</SenderID>
    <ReceiverID>PayerB</ReceiverID>
    <CORERuleVersion>2.2.0</CORERuleVersion>
    <Payload>
    <![CDATA[ ISA*00* 00 *ZZ*T000000002 *ZZ*CMS *050516*0734*^*00501*000005014*0*T*:~GS*HS*T000000002*CMS*20050516*073411*5014*X*005010X279A1~ST*270*000000001*005010X279A1~BHT*0022*13*888888880A*20080101*073411~HL*1**20*1~NM1*PR*2*CMS*****PI*CMS~HL*2*1*21*1~NM1*1P*2*NGC*****XX*1234567893~HL*3*2*22*0~TRN*1*NG-TEST*9MEDIFAX ~NM1*IL*1*LName*Mary****MI*888888880A~DMG*D8*19010101~DTP*291*RD8*20100830-20120805~EQ*30^1^35^48^14^15^67^47^AD^AE^AG^BF^BG^10^42^45^MH^2^3~SE*13*000000001~GE*1*5014~IEA*1*000005014~
    ]]>
    </Payload>
    </ns1:COREEnvelopeRealTimeRequest>
    </soapenv:Body>
    </soapenv:Envelope>

    I would appreciate if you can assist with my question above.

    Thanks,

    Lokesh
    • HermannSW
      HermannSW
      2818 Posts
      ACCEPTED ANSWER

      Re: swaform -- using HTTP forms as generator for "Soap With Attachments"

      ‏2012-10-15T18:27:50Z  in response to SystemAdmin
      Hi Lokesh,

      you took the first backup, and did two mistakes:
      • the enfpoint in the HTTP form needs to be port 30081, not 30080
      • you deleted the line <input name="soap" ... type="hidden"> which is required by first attached service (not by second)

      Looking at your requirement with CDATA output for payload means that the file you uploaded has to consist of XML Characters only.
      If this is the case, you do not need swaform -- just use normal "convert http" action.

      Find attached stylesheet toSoap.xsl which converts the convert-http output to your required output:


       
      Hermann<myXsltBlog/> <myXsltTweets/>

      Attachments

      • SystemAdmin
        SystemAdmin
        6772 Posts
        ACCEPTED ANSWER

        Re: swaform -- using HTTP forms as generator for "Soap With Attachments"

        ‏2012-10-17T05:03:08Z  in response to HermannSW
        Hello HermannSW:

        Thanks for your response. I am sorry, I made some typo errors in my previous post.

        1) I did download backup2-swaform.zip
        2) I am using HTTP MIME Multipart message to send request NOT CDATA
        I did deploy backup2-swaform.zip and deployed in DataPower. It did work with some problems.

        I am trying to solve two problems:

        1) How can I sign HTTP MIME Multipart messsage using X509 certificate using swaform tool and verify sign message ?
        2) How can I generate Error rule and send SOAP Error message back to browser if the attachment file has virus using swaform tool ?

        I added Error Rule in swaloopback XML Firewall Service to understand if there is a problem will SOAP message be sent back with message. But, what happens is when error is generated, I don't get the SOAP error fault response back to the browser to save the file. I don't know why and how can i fix that ?

        ApplicationFault.xsl

        <?xml version="1.0" encoding="UTF-8"?>
        <xsl:stylesheet version="1.0"
        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xmlns:dp="http://www.datapower.com/extensions"
        xmlns:dpconfig="http://www.datapower.com/param/config"
        xmlns:svc="http://hets.com/mimeservice/service/v1/"
        xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/"
        extension-element-prefixes="dp"
        exclude-result-prefixes="dp dpconfig"
        xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
        <xsl:output method="xml" indent="yes" />
        <xsl:template match="/">
        <soap:Envelope>
        <soap:Body>
        <soap:Fault>
        <faultcode>soap:Client</faultcode>
        <faultstring>
        <xsl:copy-of select="dp:variable('var://service/error-message')" />
        </faultstring>
        <detail>
        <svc:ApplicationFault>
        <svc:Error>
        <svc:ErrorCode>SystemError</svc:ErrorCode>
        <svc:ErrorString><xsl:copy-of select="dp:variable('var://service/error-message')" /></svc:ErrorString>
        <svc:ErrorDetail>
        trasaction-id: <xsl:copy-of select="dp:variable('var://service/transaction-id')" />
        error-code: <xsl:copy-of select="dp:variable('var://service/error-code')" />
        error-subcode: <xsl:value-of select="dp:variable('var://service/error-subcode')" />
        </svc:ErrorDetail>
        </svc:Error>
        </svc:ApplicationFault>
        </detail>
        </soap:Fault>
        </soap:Body>
        </soap:Envelope>
        </xsl:template>
        </xsl:stylesheet>

        I generated error by uploading file EICAR_Test_Virus.txt and adding virus scan action to scan entire message using ICAP protocol in swaloopback XML Firewall Service

        EICAR_Test_Virus.txt
        X5O!P%@AP[4\PZX54(P^)7CC)7}$EICAR-STANDARD-ANTIVIRUS-TEST-FILE!$H+H*

        In the probe in swaloopback XML Firewall Service, I do see my error rule is executed and SOAP Fault Message is generated. However, the SOAP Fault is not showing in the browser.
        I really appreciate your help.

        Thanks,

        Lokesh
  • SystemAdmin
    SystemAdmin
    6772 Posts
    ACCEPTED ANSWER

    Re: swaform -- using HTTP forms as generator for "Soap With Attachments"

    ‏2013-02-27T16:41:33Z  in response to HermannSW
    Herman,
    I'm trying post SWA to the second service using url-open rather than having a static backend from the first service. I keep getting error
    illegal character '-' at line 1 of http:XXX:YY.

    Can you please tell me what could be wrong here? Below is my url-open coce

    <dp:url-open target="http://xxxx:xx/" http-method="post" response="binaryNode" content-type="$contentType" http-headers="$httpHeaders">
                                    <xsl:value-of select="dp:decode(substring(dp:radix-convert(concat('ffffff', .),16,64), 5), 'base-64')"/>
                            </dp:url-open>
    

    Thanks
    Govind
    Updated on 2014-03-25T02:38:48Z at 2014-03-25T02:38:48Z by iron-man
  • SystemAdmin
    SystemAdmin
    6772 Posts
    ACCEPTED ANSWER

    Re: swaform -- using HTTP forms as generator for "Soap With Attachments"

    ‏2013-02-27T16:41:49Z  in response to HermannSW
    Herman,
    I'm trying post SWA to the second service using url-open rather than having a static backend from the first service. I keep getting error
    illegal character '-' at line 1 of http:XXX:YY.

    Can you please tell me what could be wrong here? Below is my url-open coce

    <dp:url-open target="http://xxxx:xx/" http-method="post" response="binaryNode" content-type="$contentType" http-headers="$httpHeaders">
                                    <xsl:value-of select="dp:decode(substring(dp:radix-convert(concat('ffffff', .),16,64), 5), 'base-64')"/>
                            </dp:url-open>
    

    Thanks
    Govind
    Updated on 2014-03-25T02:38:44Z at 2014-03-25T02:38:44Z by iron-man
    • HermannSW
      HermannSW
      2818 Posts
      ACCEPTED ANSWER

      Re: swaform -- using HTTP forms as generator for "Soap With Attachments"

      ‏2013-02-28T08:38:55Z  in response to SystemAdmin
      > I keep getting error illegal character '-' at line 1 of http:XXX:YY.
      >
      Is it correct that you get this error message on the 2nd appliance?
      If so, did you configure your service for attachment processing?
      (by default "Request attachment processing mode" is set to "Strip")
      What is the request type of the 2nd appliance service?

       
      Hermann<myXsltBlog/> <myXsltTweets/> <myCE/>
  • SystemAdmin
    SystemAdmin
    6772 Posts
    ACCEPTED ANSWER

    Re: swaform -- using HTTP forms as generator for "Soap With Attachments"

    ‏2013-02-28T15:44:37Z  in response to HermannSW
    > I keep getting error illegal character '-' at line 1 of http:XXX:YY.
    >
    Is it correct that you get this error message on the 2nd appliance?
    If so, did you configure your service for attachment processing?
    (by default "Request attachment processing mode" is set to "Strip")
    What is the request type of the 2nd appliance service?

    Thanks Herman for your response.

    Yes, the error is at the second service. The attachment processing is set to allow. Request type of the second service is set to SOAP.

    When I set the content-type in the url-open to text/xml, I get the below error.
    
    10:34:20      multistep       error   141265          xxxx    0x00030001      xmlfirewall (Loopback_Testing): Parse error 10:34:20    multistep       error   141265  request xxxx    0x80c00008      xmlfirewall (Loopback_Testing): rule (Loopback_Testing_request): implied action Parse input as SOAP, attempt pipeline failed: illegal character 
    '-' at line 1 of http:
    //xxxxx/ 10:34:20        xmlparse        error   141265  request xxx     0x80e003aa      xmlfirewall (Loopback_Testing): illegal character 
    '-' at line 1 of http:
    //xxxx/
    

    , bel

    When I set the content-type to multipart/related; boundary=---------------------------4664151417711; type=text/xml I get the below error

    
    10:40:12   multistep       error   119862          xxx     0x00030001      xmlfirewall (Loopback_Testing): Parse error 10:40:12    multistep       error   119862  request xxxx    0x80c00008      xmlfirewall (Loopback_Testing): rule (Loopback_Testing_request): implied action Parse input as SOAP, attempt pipeline failed: Incomplete markup or missing document element at line 1 of http:
    //xxxx/ 10:40:12   xmlparse        error   119862  request xxxx    0x80e003aa      xmlfirewall (Loopback_Testing): Incomplete markup or missing document element at line 1 of http:
    //xxxx/
    
  • SystemAdmin
    SystemAdmin
    6772 Posts
    ACCEPTED ANSWER

    Re: swaform -- using HTTP forms as generator for "Soap With Attachments"

    ‏2013-02-28T18:14:44Z  in response to HermannSW
    Herman,
    Thanks for your effort to help me with this. I got this resolved by adding the missing quotes in the content-type multipart/related; boundary="----79797....".
    • HermannSW
      HermannSW
      2818 Posts
      ACCEPTED ANSWER

      Re: swaform -- using HTTP forms as generator for "Soap With Attachments"

      ‏2013-03-01T08:25:34Z  in response to SystemAdmin
      Hi Govind,

      good that you were able to figure out what the problem was.
      Would be nice if you could add a functionally complete code snippet of your solution here.
      Please enclose code in {​​code}...{​​code} sections and use preview to make sure your posting displays as you intend before submit.

       
      Hermann<myXsltBlog/> <myXsltTweets/> <myCE/>
  • SystemAdmin
    SystemAdmin
    6772 Posts
    ACCEPTED ANSWER

    Re: swaform -- using HTTP forms as generator for "Soap With Attachments"

    ‏2013-03-05T03:30:57Z  in response to HermannSW
    Hi Herman,
    I modified your first service to use dp:url-open to send the multipart data to the 2nd service (which scans for virus in the uploaded file). Below is the XSL which posts the multipart data.

    
    <?xml version=
    "1.0"?> <xsl:stylesheet xmlns:xsl=
    "http://www.w3.org/1999/XSL/Transform" xmlns:dp=
    "http://www.datapower.com/extensions" xmlns:dpconfig=
    "http://www.datapower.com/param/config" extension-element-prefixes=
    "dp" exclude-result-prefixes=
    "dp dpconfig" version=
    "1.0"> <dp:input-mapping href=
    "hexBinary.ffd" type=
    "ffd"/> <dp:output-mapping href=
    "hexBinary.ffd" type=
    "ffd"/> <xsl:output method=
    "text"/> <xsl:template match=
    "Conversion/hexstr"> <xsl:variable name=
    "contentType" select=
    "dp:http-request-header('Content-Type')"/> <xsl:variable name=
    "httpHeaders"> <header name=
    "Content-Type"> <xsl:value-of select=
    "concat(substring-before($contentType,'='),'=&quot;',substring-after($contentType,'='),'&quot;')"/> </header> </xsl:variable> <xsl:variable name=
    "response"> <dp:url-open target=
    "http://xxxxx/" http-method=
    "post" response=
    "binaryNode" http-headers=
    "$httpHeaders"> <xsl:value-of select=
    "dp:decode(substring(dp:radix-convert(concat('ffffff', .),16,64), 5), 'base-64')"/> </dp:url-open> <dp:set-variable name=
    "'var://context/request/result'" value=
    "dp:decode(substring(dp:radix-convert(concat('ffffff', .),16,64), 5), 'base-64')"/> </xsl:variable> <xsl:variable name=
    "resp64" select=
    "dp:binary-encode($response/result/binary/child::node())"/> <xsl:variable name=
    "result" select=
    "dp:decode($resp64,'base-64')"/> <xsl:choose> <xsl:when test=
    "contains($result,'600')"> <dp:reject>virus found</dp:reject> </xsl:when> <xsl:when test=
    "contains($result,'601')"> <dp:reject>Unable to virus scan</dp:reject> </xsl:when> </xsl:choose> </xsl:template> </xsl:stylesheet>
    


    I tested this with a simple html form I created which worked fine.

    Below is the multipart data that is being sent to the second service(retrieved from a stored context variable) from my test form

    
    
    '-----------------------------24464570528145 Content-Type: text/xml; charset=UTF-8   <?xml version=
    "1.0" encoding=
    "UTF-8"?> <env:Envelope xmlns:env=
    "http://schemas.xmlsoap.org/soap/envelope/"><env:Body/></env:Envelope> -----------------------------24464570528145 Content-Id: <file> Content-Type: text/plain   
    
    this is test rs2757 rs2757rs -----------------------------24464570528145 Content-Id: <Submit>   Submit Form -----------------------------24464570528145-- 
    '
    


    But When I test it with the actual application I see the below error in the 2nd service

    
    0130304T175527Z [mime][error] mpgw(SWAForwarder): tid(48672)[request][172.17.5.44]: Invalid MIME 
    
    package: 
    
    final boundary padded by non-LWSP.
    


    Below is the multipart data being posted to the 2nd service.
    
    
    '-----------------------------114782935826962 Content-Type: text/xml; charset=UTF-8   <?xml version=
    "1.0" encoding=
    "UTF-8"?> <env:Envelope xmlns:env=
    "http://schemas.xmlsoap.org/soap/envelope/"><env:Body/></env:Envelope> -----------------------------114782935826962 Content-Id: <uploadSelector:0:uploadedFile> Content-Type: text/plain   
    
    this is test rs2757 rs2757rs -----------------------------114782935826962 Content-Id: <org.apache.myfaces.trinidad.faces.FORM>   f1 -----------------------------114782935826962 Content-Id: <javax.faces.ViewState>   !-jzpzsncuj -----------------------------114782935826962 Content-Id: <Adf-Rich-Message>   
    
    true -----------------------------114782935826962 Content-Id: <partial>   
    
    true -----------------------------114782935826962 Content-Id: <oracle.adf.view.rich.DELTAS>   
    {uploadSelector:0:tFiles=
    {rows=1
    }
    } -----------------------------114782935826962 Content-Id: <event>   uploadSelector:0:cbUpload -----------------------------114782935826962 Content-Id: <event.uploadSelector:0:cbUpload>   <m xmlns=
    "http://oracle.com/richClient/comm"><k v=
    "type"><s>action</s></k></m> -----------------------------114782935826962 Content-Id: <oracle.adf.view.rich.PROCESS>   uploadSelector -----------------------------114782935826962-- 
    '
    


    When I look at the binary data I can see the final boundary padded with "0d 0a". I can't see any difference in the message posted by my test form and the original application.

    Can you please suggest me a way to troubleshoot this one?

    Thanks
    Govind
    • HermannSW
      HermannSW
      2818 Posts
      ACCEPTED ANSWER

      Re: swaform -- using HTTP forms as generator for "Soap With Attachments"

      ‏2013-03-05T14:32:45Z  in response to SystemAdmin
      Hi Govind,

      I did ask you to put yout code into {​code}...{​code} section which you did.
      But what we can see there is not sufficient.

      The error message "Invalid MIME package: final boundary padded by non-LWSP" is pretty clear.
      LWSP is defined in this spec:
      https://tools.ietf.org/html/rfc5234

      And apparently your generated "MIME message" does contain non-LWSP at final boundary.
      Is the quote character in your code listing part of your MIME message?
      Please take a packet capture, do a "Follow TCP Stream" and look into the message in hex display to see what DataPower complains about.

       
      Hermann<myXsltBlog/> <myXsltTweets/> <myCE/>
  • SystemAdmin
    SystemAdmin
    6772 Posts
    ACCEPTED ANSWER

    Re: swaform -- using HTTP forms as generator for "Soap With Attachments"

    ‏2013-03-08T18:20:35Z  in response to HermannSW
    Herman,
    When I forward the form-data to backend webserver, it fails parsing the multi-part data. It looks like DataPower adds double quotes around the Boundary parameter which is causing this failure. Is there a way to forward this request without the double quotes.

    Appreciate your help.

    Govind
    • HermannSW
      HermannSW
      2818 Posts
      ACCEPTED ANSWER

      Re: swaform -- using HTTP forms as generator for "Soap With Attachments"

      ‏2013-03-10T00:16:48Z  in response to SystemAdmin
      Hi Govind,

      > When I forward the form-data to backend webserver, it fails parsing the multi-part data. It looks like DataPower adds
      > double quotes around the Boundary parameter which is causing this failure. Is there a way to forward this request without the double quotes.
      >
      it seems that your stylesheet adds the double quotes in the concat(...) below:
      ...
                      <xsl:variable name="contentType" select="dp:http-request-header('Content-Type')"/>
                      <xsl:variable name="httpHeaders">
                              <header name="Content-Type">
                                      <xsl:value-of select="concat(substring-before($contentType,'='),'=&quot;',substring-after($contentType,'='),'&quot;')"/>
                              </header>
                      </xsl:variable>
      ...
      


      What is the exact value of the 'Content-Type' header in INPUT context of your stylesheet?

       
      Hermann <myXsltBlog/> <myXsltTweets/> <myCE/>
      Updated on 2014-03-25T02:38:08Z at 2014-03-25T02:38:08Z by iron-man
  • SystemAdmin
    SystemAdmin
    6772 Posts
    ACCEPTED ANSWER

    Re: swaform -- using HTTP forms as generator for "Soap With Attachments"

    ‏2013-03-11T19:17:12Z  in response to HermannSW
    Herman,
    Here is the code which sets the content-type header.
    
    <xsl:variable name=
    "origContentType" select=
    "dp:variable('var://service/original-content-type')"/> <xsl:variable name=
    "contentType" select=
    "regexp:replace($origContentType,'&quot;','g','')"/> <xsl:message dp:priority=
    "info"> <xsl:value-of select=
    "$contentType"/> </xsl:message> <dp:set-http-request-header name=
    "'Content-Type'" value=
    "$contentType"/> <dp:freeze-headers/>
    


    The content-type I see in the probe after the last action is multipart/form-data; boundary="---------------------------15890672924370".
    This is the content-type of my input context as well.

    The content-type I see in the log is multipart/form-data; boundary=---------------------------15890672924370

    I can successfully use url-open to post the data to the webserver with the content-type multipart/form-data; boundary=---------------------------15890672924370.
    But when I use static backend to forward the post data, the webserver fails to parse it.

    I looked at the java class which parses the multipart data,
    it does not take care of the double quotes around the boundary.
    It seems to a bug in the parser implementation. I do not have a way to run capture.

    I would like to know if datapower automatically adds the double quotes around the boundary
    and is there a way to remove this.

    Thanks.