Topic
  • 6 replies
  • Latest Post - ‏2012-07-06T12:03:29Z by Daviiid
Daviiid
Daviiid
288 Posts

Pinned topic url-open SMTP crash DP

‏2012-07-04T12:57:56Z |
Hello

I have configured a WSP for send mail with file attachment. The message send by client is a soap with attachment (MTOM).

I use the styleSheet below to send mail. When the fileAttachment is too big (> 4 Mo) the dataPower crash!

Hermann, any idea? :o)


<xsl:stylesheet version=
"1.0" xmlns:xsl=
"http://www.w3.org/1999/XSL/Transform" xmlns:dp=
"http://www.datapower.com/extensions" xmlns:mime=
"urn:iso:identified-organization:dod:internet:mail:mixer" xmlns:func=
"http://exslt.org/functions" extension-element-prefixes=
"dp" exclude-result-prefixes=
"func mime"> <xsl:output omit-xml-declaration=
"yes" cdata-section-elements=
"corpsDuMessageHtml"/> <!-- some constants --> <xsl:variable name=
"CRLF" select=
"'&#13;&#10;'"/> <xsl:variable name=
"DDASH" select=
"'--'"/> <xsl:variable name=
"QUOT" select=
"'&quot;'"/> <!-- permet de mettre au bon format (base 64) la piece jointe. 76 caractere sur chaque ligne --> <func:function name=
"mime:split64"> <xsl:param name=
"str"/> <func:result> <xsl:value-of select=
"concat(substring($str,1,76),$CRLF)"/> <xsl:

if test=
"string-length($str) > 76"> <xsl:value-of select=
"mime:split64(substring($str,77))"/> </xsl:if> </func:result> </func:function> <xsl:template match=
"/"> <!-- recuperation de l
'emetteur--> <xsl:variable name=
"emetteur" select=
"//emetteur"/> <!-- recuperation de l
'objet du message--> <xsl:variable name=
"objetMessage" select=
"//objetMessage"/> <!-- recuperation du html dans la balise body --> <xsl:variable name=
"corpsDuMessageHtml"> <xsl:value-of disable-output-escaping=
"yes" select=
"/*[local-name()='Envelope']/*[local-name()='Body']/*[local-name()='SendMail']/*[local-name()='Mail']/*[local-name()='corpsDuMessageHtml']"/> </xsl:variable> <!-- on prepare le corps du message --> <xsl:variable name=
"serializedCorpsDuMessageHtml"> <dp:serialize select=
"$corpsDuMessageHtml" omit-xml-decl=
"yes"/> </xsl:variable> <!-- recuperation des informations sur la piece jointe si elle existe --> <xsl:variable name=
"manifest" select=
"dp:variable('var://context/INPUT/attachment-manifest')"/> <!--recuperation du fileName du fichier si piece jointe presente--> <xsl:variable name=
"filename" select=
"$manifest/*[local-name()='manifest']/*[local-name()='attachments']/*[local-name()='attachment']/*[local-name()='uri']"/> <dp:set-variable name=
"'var://context/GLOBAL/filename'" value=
"$filename"/> <!--preparation pour recuperer la piece jonte--> <xsl:variable name=
"pieceJointe"> <dp:url-open target=
"{concat($filename,'?Encode=base64')}"/> </xsl:variable> <!-- on boucle sur le nombre de destinataire : un destinataire = un mail--> <soap:Envelope xmlns:soap=
"http://www.w3.org/2003/05/soap-envelope" xmlns:ns=
"http://smabtp.fr/service/transverse/MailService/1.0/"> <soap:Header/> <soap:Body> <ns:SendMailResponse> <listeDestinataire> <xsl:for-each select=
"/*[local-name()='Envelope']/*[local-name()='Body']/*[local-name()='SendMail']/*[local-name()='Mail']/*[local-name()='Listedestinataires']/*[local-name()='destinataire']"> <destinataire> <nom> <xsl:value-of select=
"."/> </nom> <!-- recuperation du destinataire--> <xsl:variable name=
"destinataire" select=
"."/> <!-- genereration d
'une limite pour séparer les partis MIME du message --> <xsl:variable name=
"boundary" select=
"dp:generate-uuid()"/> <!-- mise en varaible de l
'appel au serveur SMTP --> <xsl:variable name=
"dummy"> <dp:url-open response=
"responsecode-ignore" target=
"{concat( 
'smtp://', dp:encode(
'pluton0.smabtp.fr',
'url'), 
'/?Recpt=', dp:encode($destinataire,
'url'), 
'&amp;Sender=', dp:encode($emetteur,
'url'), 
'&amp;Subject=', dp:encode($objetMessage,
'url'), 
'&amp;MIME=true' )
}
"> <!--<xsl:choose> <xsl:when test=
"string-length($pieceJointe) > 0">--> <xsl:value-of disable-output-escaping=
"yes" select=
"concat('MIME-Version: 1.0',$CRLF)"/> <xsl:value-of disable-output-escaping=
"yes" select=
"concat('Content-type: multipart/mixed; boundary=',$QUOT)"/> <xsl:value-of disable-output-escaping=
"yes" select=
"concat($boundary,$QUOT,$CRLF)"/> <xsl:value-of disable-output-escaping=
"yes" select=
"concat($CRLF,$CRLF,$DDASH,$boundary,$CRLF)"/> <xsl:value-of disable-output-escaping=
"yes" select=
"concat('Content-type: text/html',$CRLF,$CRLF)"/> <xsl:value-of disable-output-escaping=
"yes" select=
"$serializedCorpsDuMessageHtml"/> <xsl:value-of disable-output-escaping=
"yes" select=
"concat($CRLF,$DDASH,$boundary,$CRLF)"/> <!-- preparation de la piece jointe si elle existe--> <xsl:

if test=
"string-length($pieceJointe) > 0 "> <xsl:value-of disable-output-escaping=
"yes" select=
"concat('Content-type: ',$coty,$CRLF)"/> <xsl:value-of disable-output-escaping=
"yes" select=
"concat('Content-Transfer-Encoding: base64',$CRLF)"/> <xsl:value-of disable-output-escaping=
"yes" select=
"concat('Content-Disposition: attachment;')"/> <xsl:value-of disable-output-escaping=
"yes" select=
"concat(' filename=',$filename,';',$CRLF,$CRLF)"/> <xsl:value-of disable-output-escaping=
"yes" select=
"mime:split64($pieceJointe)"/> </xsl:if> <xsl:value-of disable-output-escaping=
"yes" select=
"concat($DDASH,$boundary,$DDASH,$CRLF)"/> <xsl:value-of disable-output-escaping=
"yes" select=
"concat('epilogue',$CRLF)"/> </dp:url-open> </xsl:variable> <xsl:choose> <xsl:when test=
"$dummy//url-open//statuscode"> <codeRetour> <xsl:value-of select=
"$dummy//url-open//statuscode"/> </codeRetour> <libelleRetour> <xsl:value-of select=
"$dummy//url-open//errorstring"/> </libelleRetour> </xsl:when> <xsl:otherwise> <codeRetour>00</codeRetour> </xsl:otherwise> </xsl:choose> </destinataire> </xsl:for-each> </listeDestinataire> </ns:SendMailResponse> </soap:Body> </soap:Envelope> <!--<soap:Envelope xmlns:soap=
"http://www.w3.org/2003/05/soap-envelope" xmlns:ns=
"http://smabtp.fr/service/transverse/MailService/1.0/"> <soap:Header/> <soap:Body> <ns:SendMailResponse> <xsl:

if test=
"$dummy//url-open//statuscode"> <responseCode> <xsl:value-of select=
"$dummy//url-open//statuscode"/> </responseCode> <libelleRetour> <xsl:value-of select=
"$dummy//url-open//errorstring"/> </libelleRetour> </xsl:if> </ns:SendMailResponse> </soap:Body> </soap:Envelope>--> <dp:set-variable name=
"'var://context/INPUT/attachment-manifest'" value=
"NULL"/> <dp:set-variable name=
"'var://service/mpgw/skip-backside'" value=
"'1'"/> </xsl:template> </xsl:stylesheet>
Updated on 2012-07-06T12:03:29Z at 2012-07-06T12:03:29Z by Daviiid
  • kenhygh
    kenhygh
    1575 Posts

    Re: url-open SMTP crash DP

    ‏2012-07-04T13:17:51Z  
    What do you mean 'crash'? Recycles? reboots? fails with an error? blows up and falls out of the rack?
    Ken
  • Daviiid
    Daviiid
    288 Posts

    Re: url-open SMTP crash DP

    ‏2012-07-04T13:51:38Z  
    • kenhygh
    • ‏2012-07-04T13:17:51Z
    What do you mean 'crash'? Recycles? reboots? fails with an error? blows up and falls out of the rack?
    Ken
    I don't know. The only thing i know is the datapower is unavailable for few seconds (screenShot)

    But there are errors (sorry, i didn't see before)

    wsgw (WS_HTTPS_SEND_MAIL): Out of memory
    wsgw (WS_HTTPS_SEND_MAIL): request WS_HTTPS_SEND_MAIL_Send #1 xform: 'Transforming INPUT with local:///dt01/sendMail.xsl results stored in PIPE' failed: Out of memory
    wsgw (WS_HTTPS_SEND_MAIL): Execution of 'local:///dt01/sendMail.xsl' aborted: Out of memory
  • Trey
    Trey
    225 Posts

    Re: url-open SMTP crash DP

    ‏2012-07-04T14:03:20Z  
    • Daviiid
    • ‏2012-07-04T13:51:38Z
    I don't know. The only thing i know is the datapower is unavailable for few seconds (screenShot)

    But there are errors (sorry, i didn't see before)

    wsgw (WS_HTTPS_SEND_MAIL): Out of memory
    wsgw (WS_HTTPS_SEND_MAIL): request WS_HTTPS_SEND_MAIL_Send #1 xform: 'Transforming INPUT with local:///dt01/sendMail.xsl results stored in PIPE' failed: Out of memory
    wsgw (WS_HTTPS_SEND_MAIL): Execution of 'local:///dt01/sendMail.xsl' aborted: Out of memory
    Check the temporary directory in the default domain if you have a backtrace.sent file or an error report then go ahead and open a pmr.
    Support can help review and comment on the information and how it relates to the firmware version and use of the smtp.
    The errors you extracted appear fairly straight forward.
  • Daviiid
    Daviiid
    288 Posts

    Re: url-open SMTP crash DP

    ‏2012-07-04T14:39:39Z  
    • Trey
    • ‏2012-07-04T14:03:20Z
    Check the temporary directory in the default domain if you have a backtrace.sent file or an error report then go ahead and open a pmr.
    Support can help review and comment on the information and how it relates to the firmware version and use of the smtp.
    The errors you extracted appear fairly straight forward.
    There is no log in the temporary directory in the default domain
  • HermannSW
    HermannSW
    4723 Posts

    Re: url-open SMTP crash DP

    ‏2012-07-04T20:48:30Z  
    Hi David,

    you caught me with my own recursive function not being implemenmted efficiently!

    DataPower XSLT compiler does not do tail recursion.
    Therefore divide-and-conquer technique (dac) is needed, see slide 5 of this webcast:
    http://www-01.ibm.com/support/docview.wss?uid=swg27019119

    Please use function mime:split64-dac() instead mime:split64(), see attached stylesheet.

    Here you can see that processing a 4MB input file can be done without problems:
    $ xpath++ "string-length(.)" input.xml 
    4000000
    $ 
    $ time coproc2 split64.dac.xsl input.xml http://dp5-l3:2223 -s > out
     
    real    0m0.996s
    user    0m0.014s
    sys     0m0.094s
    $ 
    $ wc out
      52632   52632 4315792 out
    $ 
    $ tail -3 out
    ****************************************************************************&​#13;
    ****************************************************************************&​#13;
    ********************************************&​#13;
    $
    


     
    Hermann <myXsltBlog/> <myXsltTweets/>
    Updated on 2014-03-25T02:52:00Z at 2014-03-25T02:52:00Z by iron-man
  • Daviiid
    Daviiid
    288 Posts

    Re: url-open SMTP crash DP

    ‏2012-07-06T12:03:29Z  
    • HermannSW
    • ‏2012-07-04T20:48:30Z
    Hi David,

    you caught me with my own recursive function not being implemenmted efficiently!

    DataPower XSLT compiler does not do tail recursion.
    Therefore divide-and-conquer technique (dac) is needed, see slide 5 of this webcast:
    http://www-01.ibm.com/support/docview.wss?uid=swg27019119

    Please use function mime:split64-dac() instead mime:split64(), see attached stylesheet.

    Here you can see that processing a 4MB input file can be done without problems:
    <pre class="java dw" data-editor-lang="java" data-pbcklang="java" dir="ltr">$ xpath++ "string-length(.)" input.xml 4000000 $ $ time coproc2 split64.dac.xsl input.xml http://dp5-l3:2223 -s > out real 0m0.996s user 0m0.014s sys 0m0.094s $ $ wc out 52632 52632 4315792 out $ $ tail -3 out ****************************************************************************&​#13; ****************************************************************************&​#13; ********************************************&​#13; $ </pre>

     
    Hermann <myXsltBlog/> <myXsltTweets/>
    Thanks Hermann

    it works :o)