Topic
IC4NOTICE: developerWorks Community will be offline May 29-30, 2015 while we upgrade to the latest version of IBM Connections. For more information, read our upgrade FAQ.
41 replies Latest Post - ‏2014-01-03T12:47:46Z by Daviiid
SystemAdmin
SystemAdmin
6772 Posts
ACCEPTED ANSWER

Pinned topic Sending emails with DataPower appliance (X150 v4.1.0.4)

‏2011-11-23T23:31:31Z |
Hi,

I have a requirement to send emails using DataPower appliance. After doing some research online, I found that I can use dp:url-open() function for this purpose but I wanted to know if someone has successfully implemented anything like this?

Also, is it possible to send email to multiple recipients using dp:url-open() function for SMTP?

I also have a requirement to add an attachment to the email along with the normal email-body, is that do-able?

If someone can point me to some documentation or provide some guidance for this that would be great....thanks...
Updated on 2013-01-07T08:56:14Z at 2013-01-07T08:56:14Z by Daviiid
  • HermannSW
    HermannSW
    4487 Posts
    ACCEPTED ANSWER

    Re: Sending emails with DataPower appliance (X150 v4.1.0.4)

    ‏2011-11-24T08:30:48Z  in response to SystemAdmin
    This spring we had a thread on smtp and attachments.

    Using forum search with "smtp attachment" this is the thread:
    https://www.ibm.com/developerworks/forums/thread.jspa?messageID=14679164&#14679164

    Hermann.
    • SystemAdmin
      SystemAdmin
      6772 Posts
      ACCEPTED ANSWER

      Re: Sending emails with DataPower appliance (X150 v4.1.0.4)

      ‏2011-12-13T23:08:46Z  in response to HermannSW
      Thanks for your response Hermann. But, I am still not clear about the following:

      1. Is it possible to send email to multiple recipients using dp:url-open() for smtp, also it is possible to have CC and BCC email recipients?
      2. Also, I am still not able to understand how to send an email with normal body and an attachment using dp:url-open() for smtp. I am able to send not plain text emails.

      Any help or examples in this regard would be highly appreciated.
    • dp_developer
      dp_developer
      31 Posts
      ACCEPTED ANSWER

      Re: Sending emails with DataPower appliance (X150 v4.1.0.4)

      ‏2011-12-14T14:18:17Z  in response to HermannSW
      Hi Hermann ,

      Can you please try and post a working XSL to send an e-mail with attachments (say an excel sheet or a pdf etc). It would be of great help .
      Thanks in advance .
      • HermannSW
        HermannSW
        4487 Posts
        ACCEPTED ANSWER

        Re: Sending emails with DataPower appliance (X150 v4.1.0.4)

        ‏2011-12-15T21:15:26Z  in response to dp_developer
        Hi Rahul,

        > Hi Hermann ,
        >
        > Can you please try and post a working XSL to send an e-mail with attachments (say an excel sheet or a pdf etc). It would be of great help .
        >
        >
        > Thanks in advance .
        >
        find the stylesheet smtp.xsl attached here.

        Find everything else (comments, screenshots, ...) in this smtp blog posting.

         
        Hermann<myXsltBlog/>

        Attachments

        • HermannSW
          HermannSW
          4487 Posts
          ACCEPTED ANSWER

          Re: Sending emails with DataPower appliance (X150 v4.1.0.4)

          ‏2011-12-15T22:24:19Z  in response to HermannSW
          Ups, this is a better link ...
          https://www.ibm.com/developerworks/mydeveloperworks/blogs/HermannSW/entry/smtp2
          • dp_developer
            dp_developer
            31 Posts
            ACCEPTED ANSWER

            Re: Sending emails with DataPower appliance (X150 v4.1.0.4)

            ‏2011-12-16T11:57:18Z  in response to HermannSW
            Thanks a ton Hermann...I put the values of my mail server and other details and it worked like a charm . Will get back incase of any other queries .
          • HermannSW
            HermannSW
            4487 Posts
            ACCEPTED ANSWER

            Re: Sending emails with DataPower appliance (X150 v4.1.0.4)

            ‏2011-12-20T07:56:43Z  in response to HermannSW
            Just for completeness, code listing of smtp.xslhttp://www.ibm.com/developerworks/forums/servlet/JiveServlet/download/1198-402725-14769696-376200/smtp.xsl below in addition to the screenshots from the blog posting:
            https://www.ibm.com/developerworks/mydeveloperworks/blogs/HermannSW/entry/smtp2

            Using coproc2 to execute smtp.xsl returns "sent".


            The email got sent to my private email address.
            This screenshot of my webmail client proves that sending the email with attachment really worked:


            smtp.xsl:
            
            <!-- DataPower sends HTML body email with binary data attachment (animated .gif).   see http:
            //ibm.com/developerworks/mydeveloperworks/blogs/HermannSW/entry/smtp2 --> <xsl:stylesheet version=
            "1.0" xmlns:xsl =
            "http://www.w3.org/1999/XSL/Transform" xmlns:dp  =
            "http://www.datapower.com/extensions" xmlns:mime=
            "urn:iso:identified-organization:dod:internet:mail:mixer" xmlns:func=
            "http://exslt.org/functions" extension-element-prefixes=
            "dp" exclude-result-prefixes=
            "func mime" > <xsl:output omit-xml-declaration=
            "yes" />     <xsl:template match=
            "/">   <!-- HTML email text --> <xsl:variable name=
            "html"> mail <b>bold</b> <i>italic</i> text<br/> <br/> <a href=
            "https://www.ibm.com/developerworks/mydeveloperworks/blogs/HermannSW/entry/sum_of_pascal_s_triangle_reciprocals10">Blog</a> entry on sum of Pascal
            's triangle reciprocals. <p></p><span style=
            "font-size: 1.5em;">∑<sup>∞</sup><sub>i=j+1</sub> <sup>1</sup>/<sub>(i choose j)</sub> = <sup>1</sup>/<sub>(j-1)</sub> ,    j≥2</span><p></p> … </xsl:variable>     <!-- get real binary data 
            
            for attaching to email (animated .gif) --> <xsl:variable name=
            "gif"> <dp:url-open target=
            "http://stamm-wilbrandt.de/chess/en/Pb7/anim.gif" response=
            "binaryNode" /> </xsl:variable>     <!-- now just send the email ... --> <xsl:value-of select=
            "mime:sendBodyWithAttachment( 
            'mailExchange', 
            'hermann@stamm-wilbrandt.de', 
            'johndoe@ibm.com', 
            'datapower.ibm.com', 
            'email with attachment 8', $html, dp:binary-encode($gif/result/binary), 
            'image/gif', 
            'anim.gif' )
            " /> </xsl:template>   <!-- some constants --> <xsl:variable name=
            "CRLF"  select=
            "'&#13;&#10;'"/> <xsl:variable name=
            "DDASH" select=
            "'--'"/> <xsl:variable name=
            "QUOT"  select=
            "'&quot;'"/>     <!-- MIME base64 data maximal line length is 76 characters --> <func:function name=
            "mime:split64"> <xsl:param name=
            "str"/>   <func:result> <xsl:value-of select=
            "concat(substring($str,1,76),$CRLF)"/>   <xsl:
            
            if test=
            "string-length($str) > 76"> <xsl:value-of select=
            "mime:split64(substring($str,77))"/> </xsl:if> </func:result> </func:function>     <!-- send email with HTML body and an attachment --> <func:function name=
            "mime:sendBodyWithAttachment"> <xsl:param name=
            "exchange"/><xsl:param name=
            "to"/><xsl:param name=
            "from"/> <xsl:param name=
            "domain"/><xsl:param name=
            "subject"/> <xsl:param name=
            "html"/> <xsl:param name=
            "att"/> <xsl:param name=
            "coty"/> <xsl:param name=
            "fname"/>     <!-- prepare HTML email body 
            
            for sending --> <xsl:variable name=
            "serializedHTMLData"> <dp:serialize select=
            "$html" omit-xml-decl=
            "yes"/> </xsl:variable>     <!-- as the name says, get us a unique MIME boundary --> <xsl:variable name=
            "boundary" select=
            "dp:generate-uuid()"/>     <!-- trick to make 
            
            this working in func:function --> <xsl:variable name=
            "dummy"> <dp:url-open response=
            "ignore" target=
            "{concat( 
            'smtp://', dp:encode($exchange,
            'url'), 
            '/?Recpt=', dp:encode($to,
            'url'), 
            '&amp;Sender=', dp:encode($from,
            'url'), 
            '&amp;Subject=', dp:encode($subject,
            'url'), 
            '&amp;Domain=', dp:encode($domain,
            'url'), 
            '&amp;MIME=true' )
            }
            "  > <xsl:value-of disable-output-escaping=
            "yes" select=
            "concat('MIME-Version: 1.0',$CRLF)"/> <xsl:value-of disable-output-escaping=
            "yes" select=
            "concat('Content-type: multipart/mixed; boundary=',$QUOT)"/> <xsl:value-of disable-output-escaping=
            "yes" select=
            "concat($boundary,$QUOT,$CRLF)"/>   <xsl:value-of disable-output-escaping=
            "yes" select=
            "concat($CRLF,$CRLF,$DDASH,$boundary,$CRLF)"/>   <xsl:value-of disable-output-escaping=
            "yes" select=
            "concat('Content-type: text/html',$CRLF,$CRLF)"/> <xsl:value-of disable-output-escaping=
            "yes" select=
            "$serializedHTMLData"/> <xsl:value-of disable-output-escaping=
            "yes" select=
            "concat($CRLF,$DDASH,$boundary,$CRLF)"/>   <xsl:value-of disable-output-escaping=
            "yes" select=
            "concat('Content-type: ',$coty,$CRLF)"/> <xsl:value-of disable-output-escaping=
            "yes" select=
            "concat('Content-Transfer-Encoding: base64',$CRLF)"/> <xsl:value-of disable-output-escaping=
            "yes" select=
            "concat('Content-Disposition: attachment;')"/> <xsl:value-of disable-output-escaping=
            "yes" select=
            "concat(' filename=',$fname,';',$CRLF,$CRLF)"/>   <xsl:value-of disable-output-escaping=
            "yes" select=
            "mime:split64($att)"/> <xsl:value-of disable-output-escaping=
            "yes" select=
            "concat($DDASH,$boundary,$DDASH,$CRLF)"/> <xsl:value-of disable-output-escaping=
            "yes" select=
            "concat('epilogue',$CRLF)"/> </dp:url-open> </xsl:variable>     <func:result select=
            "'sent'"/> </func:function> </xsl:stylesheet>
            


             
            Hermann<myXsltBlog/>
            • SystemAdmin
              SystemAdmin
              6772 Posts
              ACCEPTED ANSWER

              Re: Sending emails with DataPower appliance (X150 v4.1.0.4)

              ‏2011-12-22T19:50:55Z  in response to HermannSW
              Thank you so much Hermann...it worked like a charm.

              I still have one query: can we send emails to multiple recepients, without using distribution list and also is it possible to add CC and BCC email address in the email that goes out.
              Once agian I really appreciate all the help you provided, it really made my work just a piece of cake.
            • triniman10
              triniman10
              29 Posts
              ACCEPTED ANSWER

              Re: Sending emails with DataPower appliance (X150 v4.1.0.4)

              ‏2012-02-21T20:35:16Z  in response to HermannSW
              Hi Hermann. I am new to using stylesheets and we have a requirement for a client is sending an MTOM attachment via a web service call to the DP endpoint. For an initial test would like to email the attachment to the client to prove out this capability in DataPower. I've used your stmp.xsl to prove that DP can email at attachment however I'm having trouble extending this stylesheet to include the MTOM attachment. Since the MTOM is binary as well, I thought it would be easy enough to reference the attachment value from the attachment manifest but it seems that it's require more parsing/manipulation in the stylesheet that what I'm used too. Can you provide some insight in how to work with the attachment manifest to send the attachment in an email.
              • HermannSW
                HermannSW
                4487 Posts
                ACCEPTED ANSWER

                Re: Sending emails with DataPower appliance (X150 v4.1.0.4)

                ‏2012-02-23T14:29:46Z  in response to triniman10
                Hi,

                > ... For an initial test would like to email the attachment to the client to prove out this capability in DataPower.
                > I've used your stmp.xsl to prove that DP can email at attachment
                >
                good.

                >however I'm having trouble extending this stylesheet to include the MTOM attachment. Since the MTOM is binary as well,
                > I thought it would be easy enough to reference the attachment value from the attachment manifest
                >
                correct.

                > but it seems that it's require more parsing/manipulation in the stylesheet that what I'm used too.
                > Can you provide some insight in how to work with the attachment manifest to send the attachment in an email.

                Posting on "pgm1st.xsl" seems to be what you are searching for -- it
                • determines the 1st (binary) pgm file from attachment-manifest
                • extracts it (of course response="binaryNode')
                • and sends to output -- you would need to send as part of the email ...
                https://www.ibm.com/developerworks/forums/thread.jspa?messageID=14672215#14672215

                 
                Hermann<myXsltBlog/>
                • triniman10
                  triniman10
                  29 Posts
                  ACCEPTED ANSWER

                  Re: Sending emails with DataPower appliance (X150 v4.1.0.4)

                  ‏2012-02-24T17:02:32Z  in response to HermannSW
                  Thanks and that has allowed me to parse the incoming message for the attachment and send it via email. The issue now is that the email content shows as **** BINARY NODE **** and the attachment is not readable. I somewhat combined your smtp.xsl and pgm1st.xsl to produce this result and here's what that looked like:
                  <!DOCTYPE xsl:stylesheet [
                  <!ENTITY ATTMAN "dp:variable('var://local/attachment-manifest')">
                  <!ENTITY ATTHDR "/manifest/attachments/attachment/header">
                  ]>
                  <xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns:dp="http://www.datapower.com/extensions" xmlns:mime="urn:iso:identified-organization:dod:internet:mail:mixer" xmlns:func="http://exslt.org/functions" extension-element-prefixes="dp" exclude-result-prefixes="func mime">
                  <dp:input-mapping href="store:///pkcs7-convert-input.ffd" type="ffd"/>
                  <xsl:output method="text"/>
                  <xsl:template match="/">
                  <!-- HTML email text -->
                  <xsl:variable name="html">
                  <xsl:copy-of select="."/>
                  </xsl:variable>
                  <xsl:variable name="atthdrs" select="&ATTMAN;&ATTHDR;"/>
                  <xsl:message dp:priority="debug">atthdrs: <xsl:value-of select="$atthdrs"/></xsl:message>
                  <xsl:variable name="pgm1st" select="$atthdrs
                  [1]"/>
                  <xsl:message dp:priority="debug">pgm1st: <xsl:value-of select="$pgm1st"/></xsl:message>
                  <xsl:variable name="resp">
                  <dp:url-open target="{$pgm1st/../uri}" response="binaryNode"/>
                  </xsl:variable>
                  <xsl:message dp:priority="debug">resp: <xsl:value-of select="$resp"/></xsl:message>
                  <dp:strip-attachments/>
                  <xsl:variable name="attchmnt">
                  <object>
                  <message>
                  <xsl:copy-of select="$resp/result/binary/node()"/>
                  </message>
                  </object>
                  </xsl:variable>
                  <!-- now just send the email ... -->
                  <xsl:value-of select="mime:sendBodyWithAttachment(
                  'mailServer',
                  'Recipient',
                  'Sender',
                  'Domain',
                  'Subject',
                  $html,
                  dp:binary-encode($attchmnt),
                  'image/jpeg',
                  'test.jpeg'
                  )"/>
                  </xsl:template>
                  .
                  .
                  .
                  .

                  Of course, none of my xsl:message statements display anything which tells me I'm not converting them to text?
                  Also, I know I'm butchering extending your style sheets but can you shed light on a couple of actions I'm attempting to take here:
                  1) how to read in the attachment from the binarynode to send via smtp
                  2) it seems the pgm1st.xsl is coded for a specific Content-Type (image/x-portable-greymap) and the messages I need to handle can be of any legitimate type(txt/pdf/jpg etc). How would I go about handling that variable for all types?

                  I appreciate your assistance and your patience with what I'm sure are trivial questions to more experienced developers:)
                  • HermannSW
                    HermannSW
                    4487 Posts
                    ACCEPTED ANSWER

                    Re: Sending emails with DataPower appliance (X150 v4.1.0.4)

                    ‏2012-02-28T14:38:54Z  in response to triniman10
                    > ...
                    > Of course, none of my xsl:message statements display anything which tells me I'm not converting them to text?
                    If you can see "***BINARY NODE***" then you can use "dp:binary-encode(...)" to log the value base64 encoded to the log.
                    That does work for any, also binary, data.

                    > Also, I know I'm butchering extending your style sheets but can you shed light on a couple of actions I'm attempting to take here:
                    > 1) how to read in the attachment from the binarynode to send via smtp
                    See getgif.xsl sample on slide 18 from this WSTE webcast:
                    http://www-01.ibm.com/support/docview.wss?uid=swg27022979

                    > 2) it seems the pgm1st.xsl is coded for a specific Content-Type (image/x-portable-greymap) and the messages I need to handle can be of any legitimate type(txt/pdf/jpg etc). How would I go about handling that variable for all types?
                    >
                    You can select by name alternatively, just query "dp:variable('var://local/attachment-manifest')", does not need to be by content type.

                     
                    Hermann<myXsltBlog/>
                    • triniman10
                      triniman10
                      29 Posts
                      ACCEPTED ANSWER

                      Re: Sending emails with DataPower appliance (X150 v4.1.0.4)

                      ‏2012-02-29T18:04:04Z  in response to HermannSW
                      Hi Hermann, Thanks for the response and in looking at those stylesheets I see where my problem lies. I'm using your pgm1st.xsl to parse the attachment manifest and fetch the attachment thru url-open smf in the Execution Trace of Binary Node I get a response of -1 instead of a 200. Here's the snippet of the pgms1st.xsl where I'm doing the url-open. Also, I'm attaching my attachment-manifest nodeset.

                      <xsl:variable name="atthdrs" select="&ATTMAN;&ATTHDR;"/>
                      <xsl:variable name="pgm1st" select="$atthdrs"/>
                      <xsl:variable name="resp">
                      <dp:url-open target="{$pgm1st/../uri}" response="binaryNode"/></xsl:variable>
                      <xsl:variable name="encodedData" select="dp:binary-encode($resp/result/binary/node())"/>

                      The Nodeset from the Execution Trace Response shows the following:

                      <result>
                      <binary>***BINARY NODE***</binary>
                      <responsecode>-1</responsecode>
                      </result>

                      If it helps any, I'm using SoapUI to send SOAP messages with a jpg attachment to WSP hosting the wsdl I'm consuming.

                      So, it seems that I'm still not fetching attachment from SOAP Message successfully thus not able to send via email. Thoughts
                      • HermannSW
                        HermannSW
                        4487 Posts
                        ACCEPTED ANSWER

                        Re: Sending emails with DataPower appliance (X150 v4.1.0.4)

                        ‏2012-03-01T11:00:32Z  in response to triniman10
                        Please post the content of "var://local/attachment-manifest" from your Probe here.
                        In case you really send attachments it should look like shown here:
                        https://www.ibm.com/developerworks/forums/thread.jspa?messageID=14672215&#14672211

                        Your "$pgm1st" will result in a node-set of all attachments.
                        In case you really have attachments received (did you set Attachment Processing to "Allow"?),
                        you may try to select the first attachment only by
                        <xsl:variable name="pgm1st" select="$atthdrs[1]"/>

                         
                        Hermann<myXsltBlog/>
                        • Daviiid
                          Daviiid
                          282 Posts
                          ACCEPTED ANSWER

                          Re: Sending emails with DataPower appliance (X150 v4.1.0.4)

                          ‏2012-04-26T06:37:26Z  in response to HermannSW
                          Hi Hermann

                          Is it still impossible to send mail to multiple recipient?
                          • HermannSW
                            HermannSW
                            4487 Posts
                            ACCEPTED ANSWER

                            Re: Sending emails with DataPower appliance (X150 v4.1.0.4)

                            ‏2012-04-26T08:56:57Z  in response to Daviiid
                            Hi Daviid,
                            >
                            > Is it still impossible to send mail to multiple recipient?
                            >
                            yes, from my above posting:

                            > Sorry, that is not possible -- you can only specify one recipient:
                            > http://publib.boulder.ibm.com/infocenter/wsdatap/4mt/index.jsp?topic=%2Fcom.ibm.dp.xb.doc%2Fextensionfunctions43.htm&resultof=%22url-open%22%20%22smtp%22
                            >

                            The reason is that a multiple dp:url-open's would be needed if you have CCs.
                            The simplest workaround is to send to an electronic mailing list -- just define your CCs in a list, and the single dp:url-open() is sufficient.

                            Since this is not in the product, adding CC support is a new feature and therefore will never happen on existing firmwares (3.8.0-4.0.2).

                            Two days ago firmware V5.0 has been officially announced, but I am not aware that CC has been added:
                            http://www-01.ibm.com/common/ssi/cgi-bin/ssialias?infotype=AN&subtype=CA&htmlfid=897/ENUS112-064&appname=USN

                             
                            Hermann<myXsltBlog/> <myXsltTweets/>
                    • triniman10
                      triniman10
                      29 Posts
                      ACCEPTED ANSWER

                      Re: Sending emails with DataPower appliance (X150 v4.1.0.4)

                      ‏2012-05-23T22:05:11Z  in response to HermannSW
                      Hi Hermann,
                      I apologize for the delay between my posts/responses but I was pulled off on another project that took me away from the task of sending emails with attachments thru DataPower. Coincidently, I have another need for another project that requires DataPower to parse a digitally signed PDF and return it to a client as an XML populated by the parsed fields of the PDF. The PDF is a request form that will be posted from a Help Desk system. After going thru ur webcasts on Non-XML handling it looks like this is possible but seeing that I never got the emailing attachments to work, I'm in need of some more help with this task. The workflow is as follows:
                      1) DP Accept Soap with Attachment type pdf.
                      2) Parse PDF and build xml based on a schema with populated fields from PDF
                      3) Call Web Service Operation to post XML to Web Service.

                      Right now I'm testing it with XFW chained to a WSP where the XFW accepts the SwA and then after processing the pdf to xml calls the WSP endpoint for the necessary WS Operation to post data to web service.

                      I appreciate your advice on what how to parse to PDF and populate XML accordingly. Thanks!
                      • HermannSW
                        HermannSW
                        4487 Posts
                        ACCEPTED ANSWER

                        Re: Sending emails with DataPower appliance (X150 v4.1.0.4)

                        ‏2012-05-24T10:40:27Z  in response to triniman10
                        Hi Burt,

                        > 2) Parse PDF and build xml based on a schema with populated fields from PDF
                        >
                        that is a tough project, even when doing it in languages other than XSLT.

                        PDF spec from Adobe is REALLY BIG, even the first version of the spec has 756 pages ...
                        http://www.adobe.com/devnet/pdf/pdf_reference.html

                        What I know form the past is, that nearly always binary data is present in PDF files.
                        And sometimes even the complete page content is embedded as binary data which prevents reading anything without OCR software.

                        If on the oher side "your" PDFs are more or less like "normal text" (some binary data in between does not hurt) you may be able to retrieve the field values.

                        An alternative would be to extract the "application/pdf" type PDF by a fetch action from the SWA input file and send it to a webservice outside DataPower doing the XML field extraction
                        (such a service would be able to make use of libraries for parsing PDF from Adobe).

                         
                        Hermann<myXsltBlog/> <myXsltTweets/>
                        • triniman10
                          triniman10
                          29 Posts
                          ACCEPTED ANSWER

                          Re: Sending emails with DataPower appliance (X150 v4.1.0.4)

                          ‏2012-05-24T20:38:45Z  in response to HermannSW
                          Thanks Hermann! One of our Unix Admins wrote a pl script to parse the binary and build an html page with certain values from the PDF. So, it seems that the effort to move that functionality to DataPower may be too great with our deadlines. Although, if you have a stylesheet example of what parsing a PDF with more or less normal text, I would love to take a look at it.

                          v/r
                          Burt
                          • HermannSW
                            HermannSW
                            4487 Posts
                            ACCEPTED ANSWER

                            Re: Sending emails with DataPower appliance (X150 v4.1.0.4)

                            ‏2012-05-24T20:44:33Z  in response to triniman10
                            Please attach a small sample PDF with fields as in your target system.
                            Then we can just look into and decide whether its doable or not.

                             
                            Hermann<myXsltBlog/> <myXsltTweets/>
                            • triniman10
                              triniman10
                              29 Posts
                              ACCEPTED ANSWER

                              Re: Sending emails with DataPower appliance (X150 v4.1.0.4)

                              ‏2012-05-24T21:58:18Z  in response to HermannSW
                              I'll email it to you.
                            • triniman10
                              triniman10
                              29 Posts
                              ACCEPTED ANSWER

                              Re: Sending emails with DataPower appliance (X150 v4.1.0.4)

                              ‏2012-05-24T22:15:53Z  in response to HermannSW
                              I just sent you an email with a sample xml and sample pdf. Please let me know when you received it. Thanks!
                              • HermannSW
                                HermannSW
                                4487 Posts
                                ACCEPTED ANSWER

                                Re: Sending emails with DataPower appliance (X150 v4.1.0.4)

                                ‏2012-05-24T23:44:15Z  in response to triniman10
                                Hi Burt,

                                I received your email -- and I am lost.
                                I opened your form, just entered my name in "Name" field and saved that document.

                                As you can see the difference is
                                • quite big
                                • and binary.

                                I have no idea on how to extract the data from that -- perhaps a callout to your admin tool is best.



                                 
                                Hermann<myXsltBlog/> <myXsltTweets/>
                                • triniman10
                                  triniman10
                                  29 Posts
                                  ACCEPTED ANSWER

                                  Re: Sending emails with DataPower appliance (X150 v4.1.0.4)

                                  ‏2012-05-25T16:49:04Z  in response to HermannSW
                                  I appreciate your input and we will try off loading this to the unix program as I believe we can get that to work. Thanks Hermann!
                                • triniman10
                                  triniman10
                                  29 Posts
                                  ACCEPTED ANSWER

                                  Re: Sending emails with DataPower appliance (X150 v4.1.0.4)

                                  ‏2012-06-01T16:28:34Z  in response to HermannSW
                                  Hi Hermann, in trying the approach of offloading the parsing of a pdf to cgi parser in Unix I have to create a "facade" wsdl which an application will consume to push the pdf to cgi thru DataPower. The application has the following limitations on what kind of wsdl is can consume:

                                  SOAP-encoded arrays and SOAP-encoded structures are not supported. This
                                  means that RPC-encoded and document-encoded web services with complex
                                  input or output parameters do not work. For example, Amazon Web Services
                                  API and Google Web Services API fall into this category.
                                  All operations should be of one kind—that is, all rpc encoded or all doc/literal.
                                  Only SOAP operations are considered. MIME and HTTP operations are ignored.
                                  Overloaded operations are not allowed.
                                  Both input and output should be present; one-way messaging is not allowed.
                                  A WSDL file cannot have both a <wsdl:include> and a <wsdl:types> element.

                                  See attached for the wsdl I've created but the client still has trouble loading it into their system. What I need help with is developing the wsdl to include Soap with Attachments as I'm hoping it can be use in the Soap operations and not fall into one of the limitations. I'm attaching my last version of the wsdl I created for your review
                                  A second need is to fetch the attachment from the message and then inject it inline the HTTP header as the cgi backend program will only recognize it as a name/value pair in the header. Currently the CGI program parses the pdf being uploaded to a web page and when we look at those headers here's what it looks like:
                                  POST /cgi-bin/parse_2875.pl HTTP/1.1
                                  Accept: image/jpeg, application/x-ms-application, image/gif,
                                  application/xaml+xml, image/pjpeg, application/x-ms-xbap,
                                  application/vnd.ms-excel, application/vnd.ms-powerpoint, application/msword,
                                  application/vnd.xfdl; version=7.6.1.333, /
                                  Referer: https://hostname/cgi-bin/parse_2875.pl
                                  Accept-Language: en-US
                                  User-Agent: Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.1; Trident/4.0;
                                  SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; .NET4.0C;
                                  .NET4.0E; InfoPath.3)
                                  Content-Type: multipart/form-data;
                                  boundary=---------------------------7dc37f1d2f03cc
                                  Accept-Encoding: gzip, deflate
                                  Host: hostname
                                  Content-Length: 9453237
                                  Connection: Keep-Alive
                                  Cache-Control: no-cache
                                  -----------------------------7dc37f1d2f03cc
                                  Content-Disposition: form-data; name="DD2875_return"
                                  html
                                  -----------------------------7dc37f1d2f03cc
                                  Content-Disposition: form-data; name="DD2875_file";
                                  filename="\\Filelocation\DD2875.pdf"
                                  Content-Type: application/pdf

                                  So, the idea was to mimic these headers in DP and pass it on to the program that way. Is this possible to inject the pdf attachment in the HTTP Headers as binary64encoded inline
            • Daviiid
              Daviiid
              282 Posts
              ACCEPTED ANSWER

              Re: Sending emails with DataPower appliance (X150 v4.1.0.4)

              ‏2013-01-02T09:21:08Z  in response to HermannSW
              Hello Hermann

              Can you test your xslt with another subject. For exemple : 'email with accent ééé'
              • Daviiid
                Daviiid
                282 Posts
                ACCEPTED ANSWER

                Re: Sending emails with DataPower appliance (X150 v4.1.0.4)

                ‏2013-01-02T09:53:42Z  in response to Daviiid
                Hello again Hermann

                I found that

                MIME also specifies rules for encoding non-ASCII characters in email message headers, such as "Subject:", allowing these header fields to contain non-English characters
                 
                For example,
                Subject: =?iso-8859-1?Q?=A1Hola,_se=F1or!?=
                is interpreted as "Subject: ¡Hola, señor!".
                The encoded-word format is not used for the names of the headers (for example Subject). These header names are always in English in the raw message. When viewing a message with a non-English email client, the header names are usually translated by the client
                


                Is it possible to implement that in a url-open and force the subject charset?
                Updated on 2014-03-25T02:43:55Z at 2014-03-25T02:43:55Z by iron-man
                • HermannSW
                  HermannSW
                  4487 Posts
                  ACCEPTED ANSWER

                  Re: Sending emails with DataPower appliance (X150 v4.1.0.4)

                  ‏2013-01-02T23:54:11Z  in response to Daviiid
                  Hi David,

                  what you need is recursive function "mime:qp-encode-iso88591()".
                  That recursive function uses divide and conquer technique (as in [1]).
                  Then, at character level you need to call "func:codepoint()" from slide 11 of webcast [2] (attached there).
                  Finally you need to map the codepoints to quoted printable as described here.
                  You can do so by attached mapping file, which maps each value for codepoints 1-255 (0x01-0xFF).
                  Finally do not forget to split the concatenated result to lines of at most 76 characters lengths (eg. by the method in [1]).

                  For iso-8859-1 the map does need 255 entries because the Non-ASCII Latin-1 characters have codepoints 0x80-0xFF:
                  http://en.wikibooks.org/wiki/Unicode/Character_reference/0000-0FFF

                  For other encodings the mapping just needs to have more entries.

                  If you are done with that, please attach your result here.

                  [1] https://www.ibm.com/developerworks/mydeveloperworks/blogs/HermannSW/entry/doing_recursion_right17
                  [2] http://www-01.ibm.com/support/docview.wss?uid=swg27022979

                   
                  Hermann<myXsltBlog/> <myXsltTweets/>

                  Attachments

                  • Daviiid
                    Daviiid
                    282 Posts
                    ACCEPTED ANSWER

                    Re: Sending emails with DataPower appliance (X150 v4.1.0.4)

                    ‏2013-01-03T09:53:09Z  in response to HermannSW
                    Hello Hermann,

                    Thanks for your quick response.

                    I'm going to try this but for me it's a bypass solution.

                    I read that it's possible to specifically set the encoding of different MIME part.

                    It's possible to do that whith setting charset in content-type

                    <xsl:value-of disable-output-escaping="yes" select="concat('MIME-Version: 1.0',$CRLF)"/>
                    <xsl:value-of disable-output-escaping="yes" select="concat('Content-type: multipart/mixed; charset=utf-8; boundary=',$QUOT)"/>
                    


                    If i did not set charset=utf-8, caracters are interpreted in iso-8859-1 (defaut encoding of our company smtp server)

                    And in smtp protocol, it's also possible de set the encoding of the subject like this

                    Subject: =?iso-8859-1?Q?é?=
                    


                    I imagine, my problem sould be resolved if i can set this

                    Subject: =?utf-8?Q?é?=
                    


                    This is avoid to use recursive function mime:qp-encode-iso88591()

                    In addition, if i use your solution, the MIME message would be with iso-8859-1 encoding for subject and utf-8 for the body.

                    What do you think about that?
                    Updated on 2014-03-25T02:43:33Z at 2014-03-25T02:43:33Z by iron-man
                  • Daviiid
                    Daviiid
                    282 Posts
                    ACCEPTED ANSWER

                    Re: Sending emails with DataPower appliance (X150 v4.1.0.4)

                    ‏2013-01-03T10:33:42Z  in response to HermannSW
                    Hello again Hermann

                    My English is not very well and i think i created a confusion.

                    My problem is only on the subject of the mail because this is the only part of the message where it's impossible to explicitly specified the encoding for smtp server.

                    At the opposite, the differents parts of mime message can be explicitly set with a charset in content-type. (this is usefull for smtp server when the defautl encoding is iso-8859-1 and these part is sent in utf-8)

                    I hope you understan my problem with my FrenchEnglish..

                    Daviiid
                    • HermannSW
                      HermannSW
                      4487 Posts
                      ACCEPTED ANSWER

                      Re: Sending emails with DataPower appliance (X150 v4.1.0.4)

                      ‏2013-01-04T13:49:45Z  in response to Daviiid
                      Hi David,

                      what spec do you refer to?
                      In 7.1.1 from MIME spec only iso encodings and ascii are listed:
                      http://www.ietf.org/rfc/rfc1521.txt

                      And quoted-printable encoding is intended to transfer 8bit data over 7bit lines.

                      I cannot find the "=?...?=" notation in Subject line you refer to.
                      And it seems that only 8bit encodings are supported, which UTF-8 is not.

                       
                      Hermann<myXsltBlog/> <myXsltTweets/>
                      • Daviiid
                        Daviiid
                        282 Posts
                        ACCEPTED ANSWER

                        Re: Sending emails with DataPower appliance (X150 v4.1.0.4)

                        ‏2013-01-04T14:25:18Z  in response to HermannSW
                        Hi Hermann

                        I refer to this spec Link: http://tools.ietf.org/html/rfc2047

                        An 'encoded-word' may replace a 'text' token (as defined by RFC 822)
                        in any Subject or Comments header field, any extension message
                        header field, or any MIME body part field for which the field body
                        is defined as '*text'.  An 'encoded-word' may also appear in any
                        user-defined ("X-") message or body part header field.
                        


                        The following are examples of message headers containing 'encoded-word's:
                         
                           From: =?US-ASCII?Q?Keith_Moore?= <moore@cs.utk.edu>
                           To: =?ISO-8859-1?Q?Keld_J=F8rn_Simonsen?= <keld@dkuug.dk>
                           CC: =?ISO-8859-1?Q?Andr=E9?= Pirard <PIRARD@vm1.ulg.ac.be>
                           Subject: =?ISO-8859-1?B?SWYgeW91IGNhbiByZWFkIHRoaXMgeW8=?=
                            =?ISO-8859-2?B?dSB1bmRlcnN0YW5kIHRoZSBleGFtcGxlLg==?=
                        


                        We see that the charset is explicitly defined. And we see that the 'é' of André is translate by E9 and not C3A9

                        I imagine that it's possible to do this
                        CC: =?UTF-8?Q?Andr=C3A9?= Pirard <PIRARD@vm1.ulg.ac.be>
                        


                        But i found a workaround with the styleSheet i attach
                        Updated on 2014-03-25T02:43:29Z at 2014-03-25T02:43:29Z by iron-man
                        • HermannSW
                          HermannSW
                          4487 Posts
                          ACCEPTED ANSWER

                          Re: Sending emails with DataPower appliance (X150 v4.1.0.4)

                          ‏2013-01-04T22:14:09Z  in response to Daviiid
                          Hi David,

                          > But i found a workaround with the styleSheet i attach
                          >
                          your stylesheet does create a URI encoded ISO-8859-1 string, not sure how that is related to quoted-printable encoding.

                          >
                          > I refer to this spec Link: http://tools.ietf.org/html/rfc2047
                          >
                          Thanks for that link, really interesting stuff !

                          > ...
                          > We see that the charset is explicitly defined. And we see that the 'é' of André is translate by E9 and not C3A9
                          >
                          > I imagine that it's possible to do this
                          
                          CC: =?UTF-8?Q?Andr=C3A9?= Pirard <PIRARD@vm1.ulg.ac.be>
                          

                          >
                          I did test it on my private web server with email account -- you just missed the '=' before 'A9' -- '=HL' encodes just one byte.
                          This is how I simply verified that it does work by telnet (and verified that 'André' correctly showed up in my email client):
                          
                          $ telnet web23.dogado.de 25 Trying 93.90.177.145... Connected to web23.dogado.de. Escape character is 
                          '^]'. 220 web23.dogado.de ESMTP Postfix (Debian/GNU) HELO web23.dogado.de 250 web23.dogado.de MAIL FROM:me@web23.dogado.de 250 2.1.0 Ok RCPT TO:hermann@stamm-wilbrandt.de 250 2.1.5 Ok DATA 354 End data with <CR><LF>.<CR><LF> Subject: =?UTF-8?Q?Andr=C3=A9?= Pirard <PIRARD@vm1.ulg.ac.be>   This really works! . 250 2.0.0 Ok: queued as 10F761216289 ^]   telnet> quit Connection closed. $
                          

                          RFC2047 provides two encoding methods!
                          In addition to 'Q', the 'B' encoding seems to fit much better being generated from a stylesheet!
                          'B' encoding is just base64 encoding -- I verified by "telnet" that this correctly displayed Subject line in my email client:
                          
                          Subject: =?UTF-8?B?QW5kcsOp?= Pirard <PIRARD@vm1.ulg.ac.be>
                          


                          So this seems to be a really easy way to achieve UTF-8 output in subject line from within a stylesheet:
                          
                          <xsl:value-of disable-output-escaping=
                          "yes" select=
                          "concat('Subject: =?UTF-8?B?',dp:encode('André','base-64'),'?= ...')"/>
                          


                           
                          Hermann<myXsltBlog/> <myXsltTweets/>
                          • HermannSW
                            HermannSW
                            4487 Posts
                            ACCEPTED ANSWER

                            Re: Sending emails with DataPower appliance (X150 v4.1.0.4)

                            ‏2013-01-04T22:37:15Z  in response to HermannSW
                            I forgot to mention that
                            dp:encode('André','base-64')
                            


                            does the right thing just because DataPower internal encoding is UTF-8 (slide 11):
                            http://www-01.ibm.com/support/docview.wss?uid=swg27022977

                             
                            Hermann <myXsltBlog/> <myXsltTweets/>
                            Updated on 2014-03-25T02:43:25Z at 2014-03-25T02:43:25Z by iron-man
                            • Daviiid
                              Daviiid
                              282 Posts
                              ACCEPTED ANSWER

                              Re: Sending emails with DataPower appliance (X150 v4.1.0.4)

                              ‏2013-01-04T22:47:01Z  in response to HermannSW
                              Thanks a lot Hermann

                              So if i understand, i can use this code for sending email :

                              <xsl:variable name="subject">
                                  <xsl:value-of disable-output-escaping="yes" select="concat('Subject: =?UTF-8?B?',dp:encode('André','base-64'),'?=')"/>
                              </variable>
                              <dp:url-open response="ignore"
                                    target="{concat(
                                               'smtp://',
                                               dp:encode($exchange,'url'),
                                               '/?Recpt=',
                                               dp:encode($to,'url'),
                                               '&amp;Sender=',
                                               dp:encode($from,'url'),
                                               '&amp;',
                                               $subject,
                                               '&amp;Domain=',
                                               dp:encode($domain,'url'),
                                               '&amp;MIME=true'
                                             )}" 
                                  >
                                .... body ...
                               </dp:url-open>
                              
                              Updated on 2014-03-25T02:43:21Z at 2014-03-25T02:43:21Z by iron-man
                              • HermannSW
                                HermannSW
                                4487 Posts
                                ACCEPTED ANSWER

                                Re: Sending emails with DataPower appliance (X150 v4.1.0.4)

                                ‏2013-01-06T00:38:00Z  in response to Daviiid
                                Hi David,
                                >
                                > So if i understand, i can use this code for sending email :
                                > ...
                                >
                                yes.

                                Find attached modified smtp.xsl stylesheet
                                • with recursive devide-and-conquer function from this posting
                                • with interesting upside down UTF-8 encoded 'B' Subject line
                                Screenshot from my webmail client shows that Subject line is fine:


                                 
                                Hermann<myXsltBlog/> <myXsltTweets/>

                                Attachments

                                • HermannSW
                                  HermannSW
                                  4487 Posts
                                  ACCEPTED ANSWER

                                  Re: Sending emails with DataPower appliance (X150 v4.1.0.4)

                                  ‏2013-01-06T02:52:55Z  in response to HermannSW
                                  This is the snippet creating the Subject line:
                                  ...
                                                  'datapower.ibm.com',
                                                  concat(
                                                    '=?UTF-8?B?',
                                                    dp:encode('uʍop ǝpısdn ʎןǝʇǝןdɯoɔ spunos ʇɐɥʇ', 'base-64'),
                                                    '?='
                                                  ),
                                                  $html,
                                  ...
                                  


                                  But be careful, not every mail client can handle UTF-8 in Subject line properly.
                                  The upper email client (Lotus Notes, my IBM client) in below screenshot does display the Subject correctly like the previous webmail client.
                                  But I have an older webmail client that is not able to display the (UTF-8) Subject line correctly, see bottom of screenshot.
                                  You will have to make sure that all your mail clients can deal with UTF-8 encoded Subject lines correctly.


                                   
                                  Hermann <myXsltBlog/> <myXsltTweets/>
                                  Updated on 2014-03-25T02:43:17Z at 2014-03-25T02:43:17Z by iron-man
                                  • Daviiid
                                    Daviiid
                                    282 Posts
                                    ACCEPTED ANSWER

                                    Re: Sending emails with DataPower appliance (X150 v4.1.0.4)

                                    ‏2013-01-07T08:56:14Z  in response to HermannSW
                                    Hello

                                    thanks for all
                                    • RiyazShah
                                      RiyazShah
                                      52 Posts
                                      ACCEPTED ANSWER

                                      Re: Sending emails with DataPower appliance (X150 v4.1.0.4)

                                      ‏2014-01-02T20:17:34Z  in response to Daviiid

                                      Hello,

                                      I have a same scenario to send an email to someone if my backed server is failed or down.

                                      I am trying to use this code in Datapower. I am getting a error "remote error on url".

                                      <xsl:stylesheetversion="1.0"xmlns:xsl="http://www.w3.org/1999/XSL/Transform"xmlns:dp="http://www.datapower.com/extensions"extension-element-prefixes="dp">

                                      <xsl:outputomit-xml-declaration="yes"/>

                                      <xsl:templatematch="/">

                                      <dp:url-openresponse="ignore"target="{concat(

                                      'smtp://',

                                      dp:encode('mailExchange','url'),

                                      '/?Recpt=',

                                      dp:encode('abc@gmail.com','url'),

                                      '&amp;Sender=',

                                      dp:encode('xyz@gmail.com','url'),

                                      '&amp;Subject=',

                                      dp:encode('test','url'),

                                      '&amp;Domain=',

                                      dp:encode('datapower.ibm.com','url'),

                                      '&amp;MIME=false'

                                      )}">

                                      <xsl:text>12345</xsl:text>

                                      </dp:url-open>

                                      </xsl:template>

                                      </xsl:stylesheet>

                                       

                                      Can anybody help me what are the details required to generate a SMTP url.

                                       

                                      Thanks in advance.

                                       R@j

                                       

                                       

                                       

                                      Updated on 2014-01-02T20:21:52Z at 2014-01-02T20:21:52Z by RiyazShah
                                      • Daviiid
                                        Daviiid
                                        282 Posts
                                        ACCEPTED ANSWER

                                        Re: Sending emails with DataPower appliance (X150 v4.1.0.4)

                                        ‏2014-01-03T12:47:46Z  in response to RiyazShah

                                        HEllo

                                        I have no problem to send email with this styleSheet

                                        Try to use it

                                         

                                        <xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns:dp="http://www.datapower.com/extensions" extension-element-prefixes="dp">
                                            <xsl:output omit-xml-declaration="yes"/>
                                                <xsl:template match="/">
                                                        <dp:url-open response="ignore" target="{concat(
                                                                        'smtp://',
                                                                        dp:encode('smtpServer','url'),
                                                                        '/?Recpt=',
                                                                        dp:encode('mail@gmail.com','url'),
                                                                        '&amp;Sender=',
                                                                        dp:encode('mail@gmail.com','url'),
                                                                        '&amp;Subject=',
                                                                        dp:encode('test','url'),
                                                                        '&amp;Domain=',
                                                                        dp:encode('datapower.ibm.com','url'),
                                                                        '&amp;MIME=false'
                                                                        )}">
                                                                <xsl:text>12345</xsl:text>
                                                        </dp:url-open>
                                                        <dp:set-variable name="'var://service/mpgw/skip-backside'" value="'1'"/>
                                                </xsl:template>
                                        </xsl:stylesheet>
                                        

                                         

                                        Updated on 2014-01-03T12:49:27Z at 2014-01-03T12:49:27Z by Daviiid