Topic
  • 4 replies
  • Latest Post - ‏2014-03-05T17:00:06Z by PrasadGH
PrasadGH
PrasadGH
29 Posts

Pinned topic SOAP With Attachment to REST

‏2014-02-21T03:08:13Z |

Any suggestions on converting SOAP with Attachment in to DP and outbound (external clients) as REST PUT calls?

  • HermannSW
    HermannSW
    4885 Posts

    Re: SOAP With Attachment to REST

    ‏2014-02-21T08:29:54Z  

    Set "Attachment Processing" to "Allow" and do the conversion needed by your requirements in accessing the attachments by DataPower Attachment Protocol?

     

  • PrasadGH
    PrasadGH
    29 Posts

    Re: SOAP With Attachment to REST

    ‏2014-02-24T01:20:41Z  
    • HermannSW
    • ‏2014-02-21T08:29:54Z

    Set "Attachment Processing" to "Allow" and do the conversion needed by your requirements in accessing the attachments by DataPower Attachment Protocol?

     

    Thanks Herman for suggestion! Ok, from MPG --> WSP returns soap with attachment. In MPG, I can set the request as SOAP and response as JSON. In MPG, I can get attachment(s) using    <xsl:variable name="manifest" select="dp:variable('var://context/INPUT/attachment-manifest')"/> and then loop over the same to get all attachments.

    My question in this regard is, If I look at the json with attachment structure from http://en.wikipedia.org/wiki/JSON-WSP, I can convert SOAP to some JSON using some xslt transformation. Now the response is definitely the output of the conversion via the results output, but where do I need to place the extracted attachments with the specified boundary and their names and content types ? Is it just appending to json object created dynamically or can we get automatic conversion from DP?

  • HermannSW
    HermannSW
    4885 Posts

    Re: SOAP With Attachment to REST

    ‏2014-02-24T11:48:28Z  
    • PrasadGH
    • ‏2014-02-24T01:20:41Z

    Thanks Herman for suggestion! Ok, from MPG --> WSP returns soap with attachment. In MPG, I can set the request as SOAP and response as JSON. In MPG, I can get attachment(s) using    <xsl:variable name="manifest" select="dp:variable('var://context/INPUT/attachment-manifest')"/> and then loop over the same to get all attachments.

    My question in this regard is, If I look at the json with attachment structure from http://en.wikipedia.org/wiki/JSON-WSP, I can convert SOAP to some JSON using some xslt transformation. Now the response is definitely the output of the conversion via the results output, but where do I need to place the extracted attachments with the specified boundary and their names and content types ? Is it just appending to json object created dynamically or can we get automatic conversion from DP?

    Hi,

    the handling of the attachment stuff, especially the borders, is done by DataPower automagically.

    In some applications the attachments are not wanted on output, <dp:strip-attachments/> has to be used there.

    As you said just iterating over the attachments in attachment manifest and generating JSON output from that is all that is needed. I started looking at this example from the JSON-WSP link you provided:
    http://en.wikipedia.org/wiki/JSON-WSP#Attachment_service_request_example

    Find attached all files used in this demonstration, stylesheet, sample service export, sample input files and some useful tools for creating&sending SWA files by curl.

    Attached sample.swa was created by attached tools from attached cv.pdf and face.png as described below.

    Stylesheet swa2json-wsp.xsl creates the JSON part of the output, the (unmodified) attchments get copied from DataPower:

    $ cat swa2json-wsp.xsl 
    <!DOCTYPE xsl:stylesheet [ 
      <!ENTITY ATTMAN "dp:variable('var://local/attachment-manifest')"     > 
      <!ENTITY ATTURI "/manifest/attachments/attachment/uri"               > 
      <!ENTITY LF     "<xsl:text>&#10;</xsl:text>"                         > 
    ]>
    <xsl:stylesheet version="1.0"
      xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
      xmlns:dp="http://www.datapower.com/extensions"
      extension-element-prefixes="dp"
    >
      <xsl:output method="text"/>
        
      <xsl:template match="/">
    {
      "type": "jsonwsp/request",
      "version": "1.0",
      "methodname": "upload",
      "args": {
        "incoming": [
          <xsl:for-each select="&ATTMAN;&ATTURI;">
    { 
      "data": "<xsl:value-of select="."/>", 
      "name": "<xsl:value-of select="substring-after(.,'cid:')"/>"
    }
            <xsl:if test="position()!=last()">,</xsl:if>
          </xsl:for-each>
        ]
      }
    }
      </xsl:template>
      
    </xsl:stylesheet>
    $
    


    Here you can see processing of sample.swa on DataPower service via swacurl tool (that determines the boundary and populates the header variable before sending the request by curl):

    $ swacurl sample.swa http://dp3-l3:6001 
    
    
    --epoch-1393239304
    Content-Type: text/plain; charset=UTF-8
    Content-Location: root
    Content-Transfer-Encoding: binary
    
    
    {
      "type": "jsonwsp/request",
      "version": "1.0",
      "methodname": "upload",
      "args": {
        "incoming": [
          
    { 
      "data": "cid:face.png", 
      "name": "face.png"
    }
            ,
    { 
      "data": "cid:cv.pdf", 
      "name": "cv.pdf"
    }
            
        ]
      }
    }
      
    --epoch-1393239304
    Content-Type: image/png
    Content-Transfer-Encoding: binary
    Content-Id: <face.png>
    
    IHDR
    
       a���sRGB����PLTE
    
    
                          $$$888>>>AAAHHHLLLPPPRRRSSSVVVXXXYYY\\]]]fffpppttt|||~~~���������������������������������������������������������������������������������������������������������c�� pHYs
                                                           
                                                            ��tIME�7��tEXtCommentCreated with GIMPW�zIDA�E���@�;���$�j�D�=K����j��ہ���pf�<F��e�I��7�ۚC9HY��� �9`�2��f��(R��N��H)�]��u�6�ToX2��/�����#sZ��hˇ�;�HR�^IEND�B`�
    --epoch-1393239304
    Content-Type: application/pdf
    Content-Transfer-Encoding: binary
    Content-Id: <cv.pdf>
    
    %PDF-1.4 %�쏢 5 0 obj <</Length 6 0 R/Filter /FlateDecode>> stream x�+T0�3T0A(��˥d��^�ed�P�e���eff�`h$r�L���`����L"�
                                             �iendstream endobj 6 0 obj 62 endobj 4 0 obj <</Type/Page/MediaBox [0 0 612 792] /Parent 3 0 R /Resources<</ProcSet[/PDF] /ExtGState 8 0 R >> /Contents 5 0 R >> endobj 3 0 obj << /Type /Pages /Kids [ 4 0 R ] /Count 1 >> endobj 1 0 obj <</Type /Catalog /Pages 3 0 R /Metadata 9 0 R >> endobj 7 0 obj <</Type/ExtGState /OPM 1>>endobj 8 0 obj <</R7 7 0 R>> endobj 9 0 obj <</Type/Metadata /Subtype/XML/Length 1319>>stream <?xpacket begin='' id='W5M0MpCehiHzreSzNTczkc9d'?> <?adobe-xap-filters esc="CRLF"?> <x:xmpmeta xmlns:x='adobe:ns:meta/' x:xmptk='XMP toolkit 2.9.1-13, framework 1.6'> <rdf:RDF xmlns:rdf='http://www.w3.org/1999/02/22-rdf-syntax-ns#' xmlns:iX='http://ns.adobe.com/iX/1.0/'> <rdf:Description rdf:about='3fbde0fb-d545-11ee-0000-fedbbdb1aae3' xmlns:pdf='http://ns.adobe.com/pdf/1.3/' pdf:Producer='GPL Ghostscript 8.70'/> <rdf:Description rdf:about='3fbde0fb-d545-11ee-0000-fedbbdb1aae3' xmlns:xmp='http://ns.adobe.com/xap/1.0/'><xmp:ModifyDate>2014-02-24T08:53:08+01:00</xmp:ModifyDate> <xmp:CreateDate>2014-02-24T08:53:08+01:00</xmp:CreateDate> <xmp:CreatorTool>UnknownApplication</xmp:CreatorTool></rdf:Description> <rdf:Description rdf:about='3fbde0fb-d545-11ee-0000-fedbbdb1aae3' xmlns:xapMM='http://ns.adobe.com/xap/1.0/mm/' xapMM:DocumentID='3fbde0fb-d545-11ee-0000-fedbbdb1aae3'/> <rdf:Description rdf:about='3fbde0fb-d545-11ee-0000-fedbbdb1aae3' xmlns:dc='http://purl.org/dc/elements/1.1/' dc:format='application/pdf'><dc:title><rdf:Alt><rdf:li xml:lang='x-default'>Untitled</rdf:li></rdf:Alt></dc:title></rdf:Description> </rdf:RDF> </x:xmpmeta> <?xpacket end='w'?> endstream endobj 2 0 obj <</Producer(GPL Ghostscript 8.70) /CreationDate(D:20140224085308+01'00') /ModDate(D:20140224085308+01'00')>>endobj xref 0 10 0000000000 65535 f 0000000355 00000 n 0000001884 00000 n 0000000296 00000 n 0000000165 00000 n 0000000015 00000 n 0000000147 00000 n 0000000419 00000 n 0000000460 00000 n 0000000489 00000 n trailer << /Size 10 /Root 1 0 R /Info 2 0 R /ID [<DEE912C6DBF6160A7CC1DEF9754BA910><DEE912C6DBF6160A7CC1DEF9754BA910>] >> startxref 2007 %%EOF
    --epoch-1393239304--
    $
    

     

    As you can see swacurl is useful in "just sending" a .swa file to DataPower.

    $ cat swacurl 
    #!/bin/bash
    curl --data-binary @$1 -H 'Content-Type: multipart/related; type="text/xml"; boundary="'`boundary $1`'"' $2
    $ 
    $ cat boundary 
    #!/bin/bash
    if [ "$1" != "" ]; then
      input=$1
    else
      input=-
    fi
    
    sed "s/--.$/--/g" $input | grep -a "^--.*--$" | sed "s/^--\(.*\)--$/\1/g"
    $
    

     

    Last, but not least, the tools for creating a .swa sample file from several input files with correct Content-Type values in MIME parts. Java program MimeType.java needs to be compiled and .class file needs to be copied to ~/bin directory,

    $ javac -d ~/bin MimeType.java 
    $ 
    $ cat MimeType.java 
    import java.net.FileNameMap;
    import java.net.URLConnection;
    
    public class MimeType {
      public static void main(String args[]) throws Exception {
        String mimetype = URLConnection.getFileNameMap().getContentTypeFor(args[0]);
    
        System.out.println( (mimetype!=null) ? mimetype : "application/octet-stream" );
      }
    }
    $
    

     

    Bash script mimeswa makes use of MimeType and creates a .swa (Soap With Attachment) sample file with all passed files as attachments:

    $ mimeswa face.png cv.pdf > sample.swa
    $
    $ cat mimeswa 
    #!/bin/bash
    
    boundary=epoch-`date +%s`
    
    echo -e "--$boundary\xD
    Content-Type: application/soap+xml
    Content-Location: root\xD
    \xD
    <env:Envelope xmlns:env=\"http://schemas.xmlsoap.org/soap/envelope/\">
    <env:Body>
    <tii>  This is it. </tii>
    </env:Body>
    </env:Envelope>\xD"
    while [[ $# > 0 ]];
    do
    echo -ne "--$boundary\xD
    Content-Type: `java -classpath ~/bin MimeType $1`
    Content-Transfer-Encoding: binary
    Content-Id: <"$1">\xD\xA\xD\xA"`cat $1`"\xD\xA"
    shift
    done
    echo -ne "--$boundary--\xD\xA"
    $
    

     

    I did export loopback XML FW service listening on port 6001 on a 6.0.1.0 box, and verified that export swa2json-wsp.zip (contained in attachments.zip below) works well when imported on a v5 firmware box.

     

    Hermann<myXsltBlog/> <myXsltTweets/> <myCE/> <myFrameless/> <GraphvizFiddle/> <xqib/>

    Attachments

    Updated on 2014-02-24T11:59:59Z at 2014-02-24T11:59:59Z by HermannSW
  • PrasadGH
    PrasadGH
    29 Posts

    Re: SOAP With Attachment to REST

    ‏2014-03-05T17:00:06Z  
    • HermannSW
    • ‏2014-02-24T11:48:28Z

    Hi,

    the handling of the attachment stuff, especially the borders, is done by DataPower automagically.

    In some applications the attachments are not wanted on output, <dp:strip-attachments/> has to be used there.

    As you said just iterating over the attachments in attachment manifest and generating JSON output from that is all that is needed. I started looking at this example from the JSON-WSP link you provided:
    http://en.wikipedia.org/wiki/JSON-WSP#Attachment_service_request_example

    Find attached all files used in this demonstration, stylesheet, sample service export, sample input files and some useful tools for creating&sending SWA files by curl.

    Attached sample.swa was created by attached tools from attached cv.pdf and face.png as described below.

    Stylesheet swa2json-wsp.xsl creates the JSON part of the output, the (unmodified) attchments get copied from DataPower:

    <pre class="javascript dw" data-editor-lang="js" data-pbcklang="javascript" dir="ltr">$ cat swa2json-wsp.xsl <!DOCTYPE xsl:stylesheet [ <!ENTITY ATTMAN "dp:variable('var://local/attachment-manifest')" > <!ENTITY ATTURI "/manifest/attachments/attachment/uri" > <!ENTITY LF "<xsl:text>&#10;</xsl:text>" > ]> <xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns:dp="http://www.datapower.com/extensions" extension-element-prefixes="dp" > <xsl:output method="text"/> <xsl:template match="/"> { "type": "jsonwsp/request", "version": "1.0", "methodname": "upload", "args": { "incoming": [ <xsl:for-each select="&ATTMAN;&ATTURI;"> { "data": "<xsl:value-of select="."/>", "name": "<xsl:value-of select="substring-after(.,'cid:')"/>" } <xsl:if test="position()!=last()">,</xsl:if> </xsl:for-each> ] } } </xsl:template> </xsl:stylesheet> $ </pre>


    Here you can see processing of sample.swa on DataPower service via swacurl tool (that determines the boundary and populates the header variable before sending the request by curl):

    <pre class="javascript dw" data-editor-lang="js" data-pbcklang="javascript" dir="ltr">$ swacurl sample.swa http://dp3-l3:6001 --epoch-1393239304 Content-Type: text/plain; charset=UTF-8 Content-Location: root Content-Transfer-Encoding: binary { "type": "jsonwsp/request", "version": "1.0", "methodname": "upload", "args": { "incoming": [ { "data": "cid:face.png", "name": "face.png" } , { "data": "cid:cv.pdf", "name": "cv.pdf" } ] } } --epoch-1393239304 Content-Type: image/png Content-Transfer-Encoding: binary Content-Id: <face.png> IHDR a���sRGB����PLTE $$$888>>>AAAHHHLLLPPPRRRSSSVVVXXXYYY\\]]]fffpppttt|||~~~���������������������������������������������������������������������������������������������������������c�� pHYs ��tIME�7��tEXtCommentCreated with GIMPW�zIDA�E���@�;���$�j�D�=K����j��ہ���pf�<F��e�I��7�ۚC9HY��� �9`�2��f��(R��N��H)�]��u�6�ToX2��/�����#sZ��hˇ�;�HR�^IEND�B`� --epoch-1393239304 Content-Type: application/pdf Content-Transfer-Encoding: binary Content-Id: <cv.pdf> %PDF-1.4 %�쏢 5 0 obj <</Length 6 0 R/Filter /FlateDecode>> stream x�+T0�3T0A(��˥d��^�ed�P�e���eff�`h$r�L���`����L"� �iendstream endobj 6 0 obj 62 endobj 4 0 obj <</Type/Page/MediaBox [0 0 612 792] /Parent 3 0 R /Resources<</ProcSet[/PDF] /ExtGState 8 0 R >> /Contents 5 0 R >> endobj 3 0 obj << /Type /Pages /Kids [ 4 0 R ] /Count 1 >> endobj 1 0 obj <</Type /Catalog /Pages 3 0 R /Metadata 9 0 R >> endobj 7 0 obj <</Type/ExtGState /OPM 1>>endobj 8 0 obj <</R7 7 0 R>> endobj 9 0 obj <</Type/Metadata /Subtype/XML/Length 1319>>stream <?xpacket begin='' id='W5M0MpCehiHzreSzNTczkc9d'?> <?adobe-xap-filters esc="CRLF"?> <x:xmpmeta xmlns:x='adobe:ns:meta/' x:xmptk='XMP toolkit 2.9.1-13, framework 1.6'> <rdf:RDF xmlns:rdf='http://www.w3.org/1999/02/22-rdf-syntax-ns#' xmlns:iX='http://ns.adobe.com/iX/1.0/'> <rdf:Description rdf:about='3fbde0fb-d545-11ee-0000-fedbbdb1aae3' xmlns:pdf='http://ns.adobe.com/pdf/1.3/' pdf:Producer='GPL Ghostscript 8.70'/> <rdf:Description rdf:about='3fbde0fb-d545-11ee-0000-fedbbdb1aae3' xmlns:xmp='http://ns.adobe.com/xap/1.0/'><xmp:ModifyDate>2014-02-24T08:53:08+01:00</xmp:ModifyDate> <xmp:CreateDate>2014-02-24T08:53:08+01:00</xmp:CreateDate> <xmp:CreatorTool>UnknownApplication</xmp:CreatorTool></rdf:Description> <rdf:Description rdf:about='3fbde0fb-d545-11ee-0000-fedbbdb1aae3' xmlns:xapMM='http://ns.adobe.com/xap/1.0/mm/' xapMM:DocumentID='3fbde0fb-d545-11ee-0000-fedbbdb1aae3'/> <rdf:Description rdf:about='3fbde0fb-d545-11ee-0000-fedbbdb1aae3' xmlns:dc='http://purl.org/dc/elements/1.1/' dc:format='application/pdf'><dc:title><rdf:Alt><rdf:li xml:lang='x-default'>Untitled</rdf:li></rdf:Alt></dc:title></rdf:Description> </rdf:RDF> </x:xmpmeta> <?xpacket end='w'?> endstream endobj 2 0 obj <</Producer(GPL Ghostscript 8.70) /CreationDate(D:20140224085308+01'00') /ModDate(D:20140224085308+01'00')>>endobj xref 0 10 0000000000 65535 f 0000000355 00000 n 0000001884 00000 n 0000000296 00000 n 0000000165 00000 n 0000000015 00000 n 0000000147 00000 n 0000000419 00000 n 0000000460 00000 n 0000000489 00000 n trailer << /Size 10 /Root 1 0 R /Info 2 0 R /ID [<DEE912C6DBF6160A7CC1DEF9754BA910><DEE912C6DBF6160A7CC1DEF9754BA910>] >> startxref 2007 %%EOF --epoch-1393239304-- $ </pre>

     

    As you can see swacurl is useful in "just sending" a .swa file to DataPower.

    <pre class="javascript dw" data-editor-lang="js" data-pbcklang="javascript" dir="ltr">$ cat swacurl #!/bin/bash curl --data-binary @$1 -H 'Content-Type: multipart/related; type="text/xml"; boundary="'`boundary $1`'"' $2 $ $ cat boundary #!/bin/bash if [ "$1" != "" ]; then input=$1 else input=- fi sed "s/--.$/--/g" $input | grep -a "^--.*--$" | sed "s/^--\(.*\)--$/\1/g" $ </pre>

     

    Last, but not least, the tools for creating a .swa sample file from several input files with correct Content-Type values in MIME parts. Java program MimeType.java needs to be compiled and .class file needs to be copied to ~/bin directory,

    <pre class="javascript dw" data-editor-lang="js" data-pbcklang="javascript" dir="ltr">$ javac -d ~/bin MimeType.java $ $ cat MimeType.java import java.net.FileNameMap; import java.net.URLConnection; public class MimeType { public static void main(String args[]) throws Exception { String mimetype = URLConnection.getFileNameMap().getContentTypeFor(args[0]); System.out.println( (mimetype!=null) ? mimetype : "application/octet-stream" ); } } $ </pre>

     

    Bash script mimeswa makes use of MimeType and creates a .swa (Soap With Attachment) sample file with all passed files as attachments:

    <pre class="javascript dw" data-editor-lang="js" data-pbcklang="javascript" dir="ltr">$ mimeswa face.png cv.pdf > sample.swa $ $ cat mimeswa #!/bin/bash boundary=epoch-`date +%s` echo -e "--$boundary\xD Content-Type: application/soap+xml Content-Location: root\xD \xD <env:Envelope xmlns:env=\"http://schemas.xmlsoap.org/soap/envelope/\"> <env:Body> <tii> This is it. </tii> </env:Body> </env:Envelope>\xD" while [[ $# > 0 ]]; do echo -ne "--$boundary\xD Content-Type: `java -classpath ~/bin MimeType $1` Content-Transfer-Encoding: binary Content-Id: <"$1">\xD\xA\xD\xA"`cat $1`"\xD\xA" shift done echo -ne "--$boundary--\xD\xA" $ </pre>

     

    I did export loopback XML FW service listening on port 6001 on a 6.0.1.0 box, and verified that export swa2json-wsp.zip (contained in attachments.zip below) works well when imported on a v5 firmware box.

     

    Hermann<myXsltBlog/> <myXsltTweets/> <myCE/> <myFrameless/> <GraphvizFiddle/> <xqib/>

    Hi Herman,

    Thanks for the input. It gave me better understanding on how to deal with attachments. But unfortunately, requirement is a different one for me and another article of yours (SWA to mutlipart/form-data --> https://www.ibm.com/developerworks/community/forums/html/topic?id=77777777-0000-0000-0000-000014782765) gave me clue on how to solve my issues.

    Thanks..