Topic
  • 28 replies
  • Latest Post - ‏2011-08-29T06:20:56Z by charanchowdary
charanchowdary
charanchowdary
40 Posts

Pinned topic Conversion of XML file to a comma separated ASCII file

‏2011-07-27T13:59:02Z |
Any pointers on converting XML file to a comma separated ASCII file? Can it be done using XSLT?

Thanks in advance...
...
RK
Updated on 2011-08-29T06:20:56Z at 2011-08-29T06:20:56Z by charanchowdary
  • JBrennan
    JBrennan
    77 Posts

    Re: Conversion of XML file to a comma separated ASCII file

    ‏2011-07-27T14:34:18Z  
    Yes, this can be done is XSLT. Simply loop through the elements you want and append to one string with a comma between them. Since XSL variables are immutable you must think about recursion here.

    When you have the string build, simply use the xsl:value-of to output it. Be sure you have the <xsl:output method="text"/> at the beginning of the stylesheet.
  • HermannSW
    HermannSW
    4654 Posts

    Re: Conversion of XML file to a comma separated ASCII file

    ‏2011-08-02T00:35:41Z  
    Hi,

    > Any pointers on converting XML file to a comma separated ASCII file? Can it be done using XSLT?
    >
    you got the answer for XML2CSV from Jim.

    I looked up what I have and found a solution I once did for the opposite direction CSV2XML.
    The beauty of this solution is that it is a single stylesheet version based on FFD file shipped with the product.

    Sample input file "support.txt":
    
    command \ model,XA35,XS40,XI50,XB60,XM70 convert-http,yes,yes,yes,yes,yes dp:input-mapping,no,no,yes,yes,yes dp:output-mapping,no,no,yes,yes,yes
    


    Beautified (by "tidy -q -xml") XML output:
    
    <table> <row> <col>command \ model</col> <col>XA35</col> <col>XS40</col> <col>XI50</col> <col>XB60</col> <col>XM70</col> </row> <row> <col>convert-http</col> <col>yes</col> <col>yes</col> <col>yes</col> <col>yes</col> <col>yes</col> </row> <row> <col>dp:input-mapping</col> <col>no</col> <col>no</col> <col>yes</col> <col>yes</col> <col>yes</col> </row> <row> <col>dp:output-mapping</col> <col>no</col> <col>no</col> <col>yes</col> <col>yes</col> <col>yes</col> </row> </table>
    


    And this is the binary transform stylesheet "csv2xmlbin.xsl":
    
    <xsl:stylesheet version=
    "1.0" xmlns:xsl=
    "http://www.w3.org/1999/XSL/Transform" xmlns:dp=
    "http://www.datapower.com/extensions" xmlns:str=
    "http://exslt.org/strings" extension-element-prefixes=
    "dp str" > <dp:input-mapping  href=
    "store:///pkcs7-convert-input.ffd" type=
    "ffd"/>   <xsl:output omit-xml-declaration=
    "yes" />   <xsl:variable name=
    "rowsep" select=
    "'&#10;'"/> <xsl:variable name=
    "colsep" select=
    "','"/>   <xsl:template match=
    "/"> <xsl:variable name=
    "str" select=
    "dp:decode(dp:binary-encode(/object/message/node()),'base-64')"/>   <table> <xsl:for-each select=
    "str:split($str,$rowsep)"> <xsl:
    
    if test=
    "position()!=last()"> <row> <xsl:for-each select=
    "str:split(.,$colsep)"> <col><xsl:value-of select=
    "."/></col> </xsl:for-each> </row> </xsl:if> </xsl:for-each> </table> </xsl:template> </xsl:stylesheet>
    

    Hermann<myXsltBlog/>
  • charanchowdary
    charanchowdary
    40 Posts

    Re: Conversion of XML file to a comma separated ASCII file

    ‏2011-08-02T03:56:05Z  
    • HermannSW
    • ‏2011-08-02T00:35:41Z
    Hi,

    > Any pointers on converting XML file to a comma separated ASCII file? Can it be done using XSLT?
    >
    you got the answer for XML2CSV from Jim.

    I looked up what I have and found a solution I once did for the opposite direction CSV2XML.
    The beauty of this solution is that it is a single stylesheet version based on FFD file shipped with the product.

    Sample input file "support.txt":
    <pre class="jive-pre"> command \ model,XA35,XS40,XI50,XB60,XM70 convert-http,yes,yes,yes,yes,yes dp:input-mapping,no,no,yes,yes,yes dp:output-mapping,no,no,yes,yes,yes </pre>

    Beautified (by "tidy -q -xml") XML output:
    <pre class="jive-pre"> <table> <row> <col>command \ model</col> <col>XA35</col> <col>XS40</col> <col>XI50</col> <col>XB60</col> <col>XM70</col> </row> <row> <col>convert-http</col> <col>yes</col> <col>yes</col> <col>yes</col> <col>yes</col> <col>yes</col> </row> <row> <col>dp:input-mapping</col> <col>no</col> <col>no</col> <col>yes</col> <col>yes</col> <col>yes</col> </row> <row> <col>dp:output-mapping</col> <col>no</col> <col>no</col> <col>yes</col> <col>yes</col> <col>yes</col> </row> </table> </pre>

    And this is the binary transform stylesheet "csv2xmlbin.xsl":
    <pre class="jive-pre"> <xsl:stylesheet version= "1.0" xmlns:xsl= "http://www.w3.org/1999/XSL/Transform" xmlns:dp= "http://www.datapower.com/extensions" xmlns:str= "http://exslt.org/strings" extension-element-prefixes= "dp str" > <dp:input-mapping href= "store:///pkcs7-convert-input.ffd" type= "ffd"/> <xsl:output omit-xml-declaration= "yes" /> <xsl:variable name= "rowsep" select= "'&#10;'"/> <xsl:variable name= "colsep" select= "','"/> <xsl:template match= "/"> <xsl:variable name= "str" select= "dp:decode(dp:binary-encode(/object/message/node()),'base-64')"/> <table> <xsl:for-each select= "str:split($str,$rowsep)"> <xsl: if test= "position()!=last()"> <row> <xsl:for-each select= "str:split(.,$colsep)"> <col><xsl:value-of select= "."/></col> </xsl:for-each> </row> </xsl:if> </xsl:for-each> </table> </xsl:template> </xsl:stylesheet> </pre>
    Hermann<myXsltBlog/>
    Thank you Hermann for your response.

    We are actually looking for a solution to OUTPUT the below comma separated file for the INPUT xml file and vice versa. (comma separated ASCII value file to xml file).

    Appreciate any help in this regard.

    INPUT:

    <DST xml:lang="en-US">
    <jobName version="1.0">Authenticate</jobName>
    <trace>0</trace>
    <readable>N</readable>
    </DST>

    OUTPUT:

    60, 68, 83, 84, 32, 120, 109, 108, 58, 108, 97, 110, 103, 61, 34, 101, 110, 45, 85, 83, 34, 62, 13, 10, 32, 32, 32, 32, 60, 106, 111, 98, 78, 97, 109, 101, 32, 118, 101, 114, 115, 105, 111, 110, 61, 34, 49, 46, 48, 34, 62, 65, 117, 116, 104, 101, 110, 116, 105, 99, 97, 116, 101, 60, 47, 106, 111, 98, 78, 97, 109, 101, 62, 13, 10, 32, 32, 32, 32, 60, 116, 114, 97, 99, 101, 62, 48, 60, 47, 116, 114, 97, 99, 101, 62, 13, 10, 32, 32, 32, 32, 60, 114, 101, 97, 100, 97, 98, 108, 101, 62, 78, 60, 47, 114, 101, 97, 100, 97, 98, 108, 101, 62, 13, 10, 60, 47, 68, 83, 84, 62, 32, 32, 32, 32
  • JBrennan
    JBrennan
    77 Posts

    Re: Conversion of XML file to a comma separated ASCII file

    ‏2011-08-02T12:58:08Z  
    You can use what I posted to go from XML to CSV. Then Hermann gave you the reverse. TO go from CSV to XML you can also use the tokenize extenssion function.
  • HermannSW
    HermannSW
    4654 Posts

    Re: Conversion of XML file to a comma separated ASCII file

    ‏2011-08-02T13:27:21Z  
    It seems that Jim and I misunderstood your request.
    So basically you want to generate a sequence of ascii-codes of your XML file.

    This can be done by serializing the XML, and then do some fancy stuff ;-)

    I really like the 2nd line (which prevents trailing comma) in the central "concat", Oliver Becker's method of conditional selection:
    http://www.xml.org/xml/xslt_efficient_programming_techniques.pdf#page=5
    (try to find out the effect of "1 div ..." yourself if you do not already know)

    The actual conversion to character codes is done by base64 encoding the serialized XML and then recursively determine the ascii codes.

    
    $ cat asc.xml <DST xml:lang=
    "en-US"> <jobName version=
    "1.0">Authenticate</jobName> <trace>0</trace> <readable>N</readable> </DST> $ $ coproc2 asc.xsl asc.xml http:
    //cosmopolitan.boeblingen.de.ibm.com:2223; echo 60,68,83,84,32,120,109,108,58,108,97,110,103,61,34,101,110,45,85,83,34,62,10,60,106,111,98,78,97,109,101,32,118,101,114,115,105,111,110,61,34,49,46,48,34,62,65,117,116,104,101,110,116,105,99,97,116,101,60,47,106,111,98,78,97,109,101,62,10,60,116,114,97,99,101,62,48,60,47,116,114,97,99,101,62,10,60,114,101,97,100,97,98,108,101,62,78,60,47,114,101,97,100,97,98,108,101,62,10,60,47,68,83,84,62 $ $ cat asc.xsl <xsl:stylesheet version=
    "1.0" xmlns:xsl=
    "http://www.w3.org/1999/XSL/Transform" xmlns:dp=
    "http://www.datapower.com/extensions" xmlns:func=
    "http://exslt.org/functions" extension-element-prefixes=
    "dp" > <xsl:output omit-xml-declaration=
    "yes" /> <func:function name=
    "func:b64-to-dec"> <xsl:param name=
    "str"/>   <xsl:choose> <xsl:when test=
    "not($str)"/>   <xsl:otherwise> <func:result select=
    " concat( dp:radix-convert(dp:substring-base64($str, 1, 1), 64, 10), substring(
    ',', 1 div (substring($str, 3) != 
    '==')), func:b64-to-dec(dp:substring-base64($str, 2)) )
    "/> </xsl:otherwise> </xsl:choose> </func:function>   <xsl:template match=
    "/"> <xsl:variable name=
    "str"> <dp:serialize omit-xml-decl=
    "yes" select=
    "."/> </xsl:variable>   <xsl:value-of select=
    "func:b64-to-dec(dp:encode($str,'base-64'))"/> </xsl:template> </xsl:stylesheet> $
    


    Hermann<myXsltBlog/>
  • JBrennan
    JBrennan
    77 Posts

    Re: Conversion of XML file to a comma separated ASCII file

    ‏2011-08-02T13:57:29Z  
    • HermannSW
    • ‏2011-08-02T13:27:21Z
    It seems that Jim and I misunderstood your request.
    So basically you want to generate a sequence of ascii-codes of your XML file.

    This can be done by serializing the XML, and then do some fancy stuff ;-)

    I really like the 2nd line (which prevents trailing comma) in the central "concat", Oliver Becker's method of conditional selection:
    http://www.xml.org/xml/xslt_efficient_programming_techniques.pdf#page=5
    (try to find out the effect of "1 div ..." yourself if you do not already know)

    The actual conversion to character codes is done by base64 encoding the serialized XML and then recursively determine the ascii codes.

    <pre class="jive-pre"> $ cat asc.xml <DST xml:lang= "en-US"> <jobName version= "1.0">Authenticate</jobName> <trace>0</trace> <readable>N</readable> </DST> $ $ coproc2 asc.xsl asc.xml http: //cosmopolitan.boeblingen.de.ibm.com:2223; echo 60,68,83,84,32,120,109,108,58,108,97,110,103,61,34,101,110,45,85,83,34,62,10,60,106,111,98,78,97,109,101,32,118,101,114,115,105,111,110,61,34,49,46,48,34,62,65,117,116,104,101,110,116,105,99,97,116,101,60,47,106,111,98,78,97,109,101,62,10,60,116,114,97,99,101,62,48,60,47,116,114,97,99,101,62,10,60,114,101,97,100,97,98,108,101,62,78,60,47,114,101,97,100,97,98,108,101,62,10,60,47,68,83,84,62 $ $ cat asc.xsl <xsl:stylesheet version= "1.0" xmlns:xsl= "http://www.w3.org/1999/XSL/Transform" xmlns:dp= "http://www.datapower.com/extensions" xmlns:func= "http://exslt.org/functions" extension-element-prefixes= "dp" > <xsl:output omit-xml-declaration= "yes" /> <func:function name= "func:b64-to-dec"> <xsl:param name= "str"/> <xsl:choose> <xsl:when test= "not($str)"/> <xsl:otherwise> <func:result select= " concat( dp:radix-convert(dp:substring-base64($str, 1, 1), 64, 10), substring( ',', 1 div (substring($str, 3) != '==')), func:b64-to-dec(dp:substring-base64($str, 2)) ) "/> </xsl:otherwise> </xsl:choose> </func:function> <xsl:template match= "/"> <xsl:variable name= "str"> <dp:serialize omit-xml-decl= "yes" select= "."/> </xsl:variable> <xsl:value-of select= "func:b64-to-dec(dp:encode($str,'base-64'))"/> </xsl:template> </xsl:stylesheet> $ </pre>

    Hermann<myXsltBlog/>
    ah yea, I misunderstood it completely. Very slick Hermann!
  • Liv2luv
    Liv2luv
    573 Posts

    Re: Conversion of XML file to a comma separated ASCII file

    ‏2011-08-02T14:22:20Z  
    • HermannSW
    • ‏2011-08-02T13:27:21Z
    It seems that Jim and I misunderstood your request.
    So basically you want to generate a sequence of ascii-codes of your XML file.

    This can be done by serializing the XML, and then do some fancy stuff ;-)

    I really like the 2nd line (which prevents trailing comma) in the central "concat", Oliver Becker's method of conditional selection:
    http://www.xml.org/xml/xslt_efficient_programming_techniques.pdf#page=5
    (try to find out the effect of "1 div ..." yourself if you do not already know)

    The actual conversion to character codes is done by base64 encoding the serialized XML and then recursively determine the ascii codes.

    <pre class="jive-pre"> $ cat asc.xml <DST xml:lang= "en-US"> <jobName version= "1.0">Authenticate</jobName> <trace>0</trace> <readable>N</readable> </DST> $ $ coproc2 asc.xsl asc.xml http: //cosmopolitan.boeblingen.de.ibm.com:2223; echo 60,68,83,84,32,120,109,108,58,108,97,110,103,61,34,101,110,45,85,83,34,62,10,60,106,111,98,78,97,109,101,32,118,101,114,115,105,111,110,61,34,49,46,48,34,62,65,117,116,104,101,110,116,105,99,97,116,101,60,47,106,111,98,78,97,109,101,62,10,60,116,114,97,99,101,62,48,60,47,116,114,97,99,101,62,10,60,114,101,97,100,97,98,108,101,62,78,60,47,114,101,97,100,97,98,108,101,62,10,60,47,68,83,84,62 $ $ cat asc.xsl <xsl:stylesheet version= "1.0" xmlns:xsl= "http://www.w3.org/1999/XSL/Transform" xmlns:dp= "http://www.datapower.com/extensions" xmlns:func= "http://exslt.org/functions" extension-element-prefixes= "dp" > <xsl:output omit-xml-declaration= "yes" /> <func:function name= "func:b64-to-dec"> <xsl:param name= "str"/> <xsl:choose> <xsl:when test= "not($str)"/> <xsl:otherwise> <func:result select= " concat( dp:radix-convert(dp:substring-base64($str, 1, 1), 64, 10), substring( ',', 1 div (substring($str, 3) != '==')), func:b64-to-dec(dp:substring-base64($str, 2)) ) "/> </xsl:otherwise> </xsl:choose> </func:function> <xsl:template match= "/"> <xsl:variable name= "str"> <dp:serialize omit-xml-decl= "yes" select= "."/> </xsl:variable> <xsl:value-of select= "func:b64-to-dec(dp:encode($str,'base-64'))"/> </xsl:template> </xsl:stylesheet> $ </pre>

    Hermann<myXsltBlog/>
    Hi Hermann,

    I was looking at it a little differently, not as good view point as yours :)

    The xml message will be converted into a hex stream and passed to this XSL (see below) which outputs ASCII codes for corresponding hex character (read from a mapping xml document)

    
    <?xml version=
    "1.0" encoding=
    "UTF-8"?> <xsl:stylesheet xmlns:xsl=
    "http://www.w3.org/1999/XSL/Transform" version=
    "1.0"> <xsl:output method=
    "xml"/>   <xsl:template match=
    "root/*/text()"> <xsl:
    
    if test=
    "string-length(.) mod 2 = 0">   <xsl:variable name=
    "csvString"> <xsl:call-template name=
    "csvGen"> <xsl:with-param name=
    "index" select=
    "1"/> <xsl:with-param name=
    "text" select=
    "."/> <xsl:with-param name=
    "strLength" select=
    "string-length(.)"/> </xsl:call-template> </xsl:variable> <out> <xsl:value-of select=
    "concat('[',$csvString,']')"/> </out> </xsl:if> </xsl:template> <xsl:template name=
    "csvGen"> <xsl:param name=
    "index"/> <xsl:param name=
    "text"/> <xsl:param name=
    "strLength"/> <xsl:choose> <xsl:when test=
    "string-length($text) = '2'"> <xsl:value-of select=
    "$text"/> </xsl:when> <xsl:otherwise> <xsl:variable name=
    "currentHex" select=
    "substring($text,$index,2)"/> <xsl:value-of select=
    "document('char-map.xml')/characterMap/character/hex[.=$currentHex]/../alt/text()"/> <xsl:text>, </xsl:text> <xsl:call-template name=
    "csvGen"> <xsl:with-param name=
    "index" select=
    "1"/> <xsl:with-param name=
    "text" select=
    "substring($text,$index+2)"/> <xsl:with-param name=
    "strLength" select=
    "string-length(substring($text,$index+2))"/> </xsl:call-template> </xsl:otherwise> </xsl:choose> </xsl:template> </xsl:stylesheet>
    


    mapping document from hex to ascii can be like this :

    
    <characterMap> <character> <ascii>A</ascii> <alt>65</alt> <hex>41</hex> </character> ... </characterMap>
    


    which produces the same output ... I liked your way though.

    Thanks.
  • Liv2luv
    Liv2luv
    573 Posts

    Re: Conversion of XML file to a comma separated ASCII file

    ‏2011-08-02T14:31:43Z  
    • Liv2luv
    • ‏2011-08-02T14:22:20Z
    Hi Hermann,

    I was looking at it a little differently, not as good view point as yours :)

    The xml message will be converted into a hex stream and passed to this XSL (see below) which outputs ASCII codes for corresponding hex character (read from a mapping xml document)

    <pre class="jive-pre"> <?xml version= "1.0" encoding= "UTF-8"?> <xsl:stylesheet xmlns:xsl= "http://www.w3.org/1999/XSL/Transform" version= "1.0"> <xsl:output method= "xml"/> <xsl:template match= "root/*/text()"> <xsl: if test= "string-length(.) mod 2 = 0"> <xsl:variable name= "csvString"> <xsl:call-template name= "csvGen"> <xsl:with-param name= "index" select= "1"/> <xsl:with-param name= "text" select= "."/> <xsl:with-param name= "strLength" select= "string-length(.)"/> </xsl:call-template> </xsl:variable> <out> <xsl:value-of select= "concat('[',$csvString,']')"/> </out> </xsl:if> </xsl:template> <xsl:template name= "csvGen"> <xsl:param name= "index"/> <xsl:param name= "text"/> <xsl:param name= "strLength"/> <xsl:choose> <xsl:when test= "string-length($text) = '2'"> <xsl:value-of select= "$text"/> </xsl:when> <xsl:otherwise> <xsl:variable name= "currentHex" select= "substring($text,$index,2)"/> <xsl:value-of select= "document('char-map.xml')/characterMap/character/hex[.=$currentHex]/../alt/text()"/> <xsl:text>, </xsl:text> <xsl:call-template name= "csvGen"> <xsl:with-param name= "index" select= "1"/> <xsl:with-param name= "text" select= "substring($text,$index+2)"/> <xsl:with-param name= "strLength" select= "string-length(substring($text,$index+2))"/> </xsl:call-template> </xsl:otherwise> </xsl:choose> </xsl:template> </xsl:stylesheet> </pre>

    mapping document from hex to ascii can be like this :

    <pre class="jive-pre"> <characterMap> <character> <ascii>A</ascii> <alt>65</alt> <hex>41</hex> </character> ... </characterMap> </pre>

    which produces the same output ... I liked your way though.

    Thanks.
    BTW, a node-set with hex stream will be produced by using hexBinary.ffd in the dp:input-mapping.

    Thanks.
  • Liv2luv
    Liv2luv
    573 Posts

    Re: Conversion of XML file to a comma separated ASCII file

    ‏2011-08-03T11:02:11Z  
    • HermannSW
    • ‏2011-08-02T13:27:21Z
    It seems that Jim and I misunderstood your request.
    So basically you want to generate a sequence of ascii-codes of your XML file.

    This can be done by serializing the XML, and then do some fancy stuff ;-)

    I really like the 2nd line (which prevents trailing comma) in the central "concat", Oliver Becker's method of conditional selection:
    http://www.xml.org/xml/xslt_efficient_programming_techniques.pdf#page=5
    (try to find out the effect of "1 div ..." yourself if you do not already know)

    The actual conversion to character codes is done by base64 encoding the serialized XML and then recursively determine the ascii codes.

    <pre class="jive-pre"> $ cat asc.xml <DST xml:lang= "en-US"> <jobName version= "1.0">Authenticate</jobName> <trace>0</trace> <readable>N</readable> </DST> $ $ coproc2 asc.xsl asc.xml http: //cosmopolitan.boeblingen.de.ibm.com:2223; echo 60,68,83,84,32,120,109,108,58,108,97,110,103,61,34,101,110,45,85,83,34,62,10,60,106,111,98,78,97,109,101,32,118,101,114,115,105,111,110,61,34,49,46,48,34,62,65,117,116,104,101,110,116,105,99,97,116,101,60,47,106,111,98,78,97,109,101,62,10,60,116,114,97,99,101,62,48,60,47,116,114,97,99,101,62,10,60,114,101,97,100,97,98,108,101,62,78,60,47,114,101,97,100,97,98,108,101,62,10,60,47,68,83,84,62 $ $ cat asc.xsl <xsl:stylesheet version= "1.0" xmlns:xsl= "http://www.w3.org/1999/XSL/Transform" xmlns:dp= "http://www.datapower.com/extensions" xmlns:func= "http://exslt.org/functions" extension-element-prefixes= "dp" > <xsl:output omit-xml-declaration= "yes" /> <func:function name= "func:b64-to-dec"> <xsl:param name= "str"/> <xsl:choose> <xsl:when test= "not($str)"/> <xsl:otherwise> <func:result select= " concat( dp:radix-convert(dp:substring-base64($str, 1, 1), 64, 10), substring( ',', 1 div (substring($str, 3) != '==')), func:b64-to-dec(dp:substring-base64($str, 2)) ) "/> </xsl:otherwise> </xsl:choose> </func:function> <xsl:template match= "/"> <xsl:variable name= "str"> <dp:serialize omit-xml-decl= "yes" select= "."/> </xsl:variable> <xsl:value-of select= "func:b64-to-dec(dp:encode($str,'base-64'))"/> </xsl:template> </xsl:stylesheet> $ </pre>

    Hermann<myXsltBlog/>
    Hi Hermann,

    Further, based on your approach, I've run some tests. At the content length of 15026 bytes and above, my datapower device was reporting a stack overflow error. Decoding base-64 for each character recursively could be the cause.

    Thanks.
  • HermannSW
    HermannSW
    4654 Posts

    Re: Conversion of XML file to a comma separated ASCII file

    ‏2011-08-03T15:40:23Z  
    • Liv2luv
    • ‏2011-08-03T11:02:11Z
    Hi Hermann,

    Further, based on your approach, I've run some tests. At the content length of 15026 bytes and above, my datapower device was reporting a stack overflow error. Decoding base-64 for each character recursively could be the cause.

    Thanks.
    >
    > Further, based on your approach, I've run some tests. At the content length of 15026 bytes and above,
    > my datapower device was reporting a stack overflow error. Decoding base-64 for each character recursively could be the cause.
    >
    Sorry, the solution I posted was the simple one.
    Yes, the recursion is the cause.

    See slide "Doing recursion right" on how the posted stylesheet could be modified:
    http://www-01.ibm.com/support/docview.wss?uid=swg27019119&aid=1#page=5

    Also see slides on "Stylesheet Profiling":
    http://www-01.ibm.com/support/docview.wss?uid=swg27019118&aid=1#page=4

    I do not have the time but would bet that your lookup solution, while working fine, may have some performance penalties.

    On one of the slides I stated
    "Prefer use of extension functions over pure XSLT solutions because of performance"
    and this is still true.
    Extension function dp:radix-convert(_,64,10) should be more efficient than looking up a charmap in XSLT.
    Perhaps you may want to do some profiling?

     
    Hermann<myXsltBlog/>
  • charanchowdary
    charanchowdary
    40 Posts

    Re: Conversion of XML file to a comma separated ASCII file

    ‏2011-08-10T19:15:13Z  
    • HermannSW
    • ‏2011-08-02T13:27:21Z
    It seems that Jim and I misunderstood your request.
    So basically you want to generate a sequence of ascii-codes of your XML file.

    This can be done by serializing the XML, and then do some fancy stuff ;-)

    I really like the 2nd line (which prevents trailing comma) in the central "concat", Oliver Becker's method of conditional selection:
    http://www.xml.org/xml/xslt_efficient_programming_techniques.pdf#page=5
    (try to find out the effect of "1 div ..." yourself if you do not already know)

    The actual conversion to character codes is done by base64 encoding the serialized XML and then recursively determine the ascii codes.

    <pre class="jive-pre"> $ cat asc.xml <DST xml:lang= "en-US"> <jobName version= "1.0">Authenticate</jobName> <trace>0</trace> <readable>N</readable> </DST> $ $ coproc2 asc.xsl asc.xml http: //cosmopolitan.boeblingen.de.ibm.com:2223; echo 60,68,83,84,32,120,109,108,58,108,97,110,103,61,34,101,110,45,85,83,34,62,10,60,106,111,98,78,97,109,101,32,118,101,114,115,105,111,110,61,34,49,46,48,34,62,65,117,116,104,101,110,116,105,99,97,116,101,60,47,106,111,98,78,97,109,101,62,10,60,116,114,97,99,101,62,48,60,47,116,114,97,99,101,62,10,60,114,101,97,100,97,98,108,101,62,78,60,47,114,101,97,100,97,98,108,101,62,10,60,47,68,83,84,62 $ $ cat asc.xsl <xsl:stylesheet version= "1.0" xmlns:xsl= "http://www.w3.org/1999/XSL/Transform" xmlns:dp= "http://www.datapower.com/extensions" xmlns:func= "http://exslt.org/functions" extension-element-prefixes= "dp" > <xsl:output omit-xml-declaration= "yes" /> <func:function name= "func:b64-to-dec"> <xsl:param name= "str"/> <xsl:choose> <xsl:when test= "not($str)"/> <xsl:otherwise> <func:result select= " concat( dp:radix-convert(dp:substring-base64($str, 1, 1), 64, 10), substring( ',', 1 div (substring($str, 3) != '==')), func:b64-to-dec(dp:substring-base64($str, 2)) ) "/> </xsl:otherwise> </xsl:choose> </func:function> <xsl:template match= "/"> <xsl:variable name= "str"> <dp:serialize omit-xml-decl= "yes" select= "."/> </xsl:variable> <xsl:value-of select= "func:b64-to-dec(dp:encode($str,'base-64'))"/> </xsl:template> </xsl:stylesheet> $ </pre>

    Hermann<myXsltBlog/>
    Thank you Hermann. It works. However my DP configuration is not able to produce a valid message for the back end AS/400 application over TCP/IP. Here is what I have.

    1. Attached word document <<AS400IntegrationSpec.doc>> has back end AS/400 integration specifications.
    2. I have configured MPG object with

    a) xform action in request rule with below xslt, which is a modified version of above posted asc.xsl to include metadata per above point 1.
    b) static back end as tcp://as400hostname:as400port
    c) Request type as xml and
    d) Response type as non-xml

    <xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns:dp="http://www.datapower.com/extensions" xmlns:func="http://exslt.org/functions" extension-element-prefixes="dp">
    <xsl:output omit-xml-declaration="yes"/>
    <func:function name="func:b64-to-dec">
    <xsl:param name="str"/>
    <xsl:choose>
    <xsl:when test="not($str)"/>
    <xsl:otherwise>
    <func:result select="
    concat(
    dp:radix-convert(dp:substring-base64($str, 1, 1), 64, 10),
    func:b64-to-dec(dp:substring-base64($str, 2))
    )"/>
    </xsl:otherwise>
    </xsl:choose>
    </func:function>
    <xsl:template match="/">
    <xsl:variable name="str">
    <dp:serialize omit-xml-decl="yes" select="."/>
    </xsl:variable>
    <xsl:variable name="XMLByteArray">
    <xsl:value-of select="func:b64-to-dec(dp:encode($str,'base-64'))"/>
    </xsl:variable>
    <xsl:message dp:priority="debug">XMLByteArray: <xsl:value-of select="$XMLByteArray"/>
    </xsl:message>
    <xsl:variable name="XMLByteArrayLength">
    <xsl:value-of select="string-length($XMLByteArray)"/>
    <!-- <xsl:value-of select="dp:variable('var://service/mpgw/request-size')"/> -->
    </xsl:variable>
    <xsl:message dp:priority="debug">XMLByteArrayLength: <xsl:value-of select="$XMLByteArrayLength"/>
    </xsl:message>
    <xsl:value-of select="1"/>
    <xsl:value-of select="$XMLByteArrayLength"/>
    <xsl:value-of select="0"/>
    <xsl:value-of select="$XMLByteArray"/>
    </xsl:template>
    </xsl:stylesheet>

    with above config AS400 application is throwing some internal error as it is unable to process the message sent from DataPower.
    Wondering about what went wrong with above xslt. Any help is welcome.
  • HermannSW
    HermannSW
    4654 Posts

    Re: Conversion of XML file to a comma separated ASCII file

    ‏2011-08-11T21:06:05Z  
    Thank you Hermann. It works. However my DP configuration is not able to produce a valid message for the back end AS/400 application over TCP/IP. Here is what I have.

    1. Attached word document <<AS400IntegrationSpec.doc>> has back end AS/400 integration specifications.
    2. I have configured MPG object with

    a) xform action in request rule with below xslt, which is a modified version of above posted asc.xsl to include metadata per above point 1.
    b) static back end as tcp://as400hostname:as400port
    c) Request type as xml and
    d) Response type as non-xml

    <xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns:dp="http://www.datapower.com/extensions" xmlns:func="http://exslt.org/functions" extension-element-prefixes="dp">
    <xsl:output omit-xml-declaration="yes"/>
    <func:function name="func:b64-to-dec">
    <xsl:param name="str"/>
    <xsl:choose>
    <xsl:when test="not($str)"/>
    <xsl:otherwise>
    <func:result select="
    concat(
    dp:radix-convert(dp:substring-base64($str, 1, 1), 64, 10),
    func:b64-to-dec(dp:substring-base64($str, 2))
    )"/>
    </xsl:otherwise>
    </xsl:choose>
    </func:function>
    <xsl:template match="/">
    <xsl:variable name="str">
    <dp:serialize omit-xml-decl="yes" select="."/>
    </xsl:variable>
    <xsl:variable name="XMLByteArray">
    <xsl:value-of select="func:b64-to-dec(dp:encode($str,'base-64'))"/>
    </xsl:variable>
    <xsl:message dp:priority="debug">XMLByteArray: <xsl:value-of select="$XMLByteArray"/>
    </xsl:message>
    <xsl:variable name="XMLByteArrayLength">
    <xsl:value-of select="string-length($XMLByteArray)"/>
    <!-- <xsl:value-of select="dp:variable('var://service/mpgw/request-size')"/> -->
    </xsl:variable>
    <xsl:message dp:priority="debug">XMLByteArrayLength: <xsl:value-of select="$XMLByteArrayLength"/>
    </xsl:message>
    <xsl:value-of select="1"/>
    <xsl:value-of select="$XMLByteArrayLength"/>
    <xsl:value-of select="0"/>
    <xsl:value-of select="$XMLByteArray"/>
    </xsl:template>
    </xsl:stylesheet>

    with above config AS400 application is throwing some internal error as it is unable to process the message sent from DataPower.
    Wondering about what went wrong with above xslt. Any help is welcome.
    > Thank you Hermann. It works. However my DP configuration is not able to produce a valid message
    > for the back end AS/400 application over TCP/IP. Here is what I have.
    >
    > 1. Attached word document <<AS400IntegrationSpec.doc>> has back end AS/400 integration specifications.
    > 2. I have configured MPG object with
    >
    > a) xform action in request rule with below xslt, which is a modified version of above posted asc.xsl to include metadata per above point 1.
    > b) static back end as tcp://as400hostname:as400port
    correct.

    > c) Request type as xml and
    > d) Response type as non-xml
    correct.

    >
    > <xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns:dp="http://www.datapower.com/extensions" xmlns:func="http://exslt.org/functions" extension-element-prefixes="dp">
    > <xsl:output omit-xml-declaration="yes"/>
    ...
    ...
    > <xsl:value-of select="1"/>
    > <xsl:value-of select="$XMLByteArrayLength"/>
    > <xsl:value-of select="0"/>
    > <xsl:value-of select="$XMLByteArray"/>
    > </xsl:template>
    > </xsl:stylesheet>
    >
    > with above config AS400 application is throwing some internal error as it is unable to process the message sent from DataPower.
    > Wondering about what went wrong with above xslt. Any help is welcome.
    >
    Either I or you does not understand your specification document correctly.

    The 4th part of a request by your spec is a "XML Byte Array", the length of which has been specified in 3rd entry "Document Byte Array Length".

    You have to send "bytes" (octets), and not a sequence of decimal numbers like you did if I am right.
    Then you do not need the conversion you asked for at all.
    You should just do <xsl:copy-of select="/" /> and be fine ...

     
    Hermann<myXsltBlog/>
  • charanchowdary
    charanchowdary
    40 Posts

    Re: Conversion of XML file to a comma separated ASCII file

    ‏2011-08-11T22:12:41Z  
    • HermannSW
    • ‏2011-08-11T21:06:05Z
    > Thank you Hermann. It works. However my DP configuration is not able to produce a valid message
    > for the back end AS/400 application over TCP/IP. Here is what I have.
    >
    > 1. Attached word document <<AS400IntegrationSpec.doc>> has back end AS/400 integration specifications.
    > 2. I have configured MPG object with
    >
    > a) xform action in request rule with below xslt, which is a modified version of above posted asc.xsl to include metadata per above point 1.
    > b) static back end as tcp://as400hostname:as400port
    correct.

    > c) Request type as xml and
    > d) Response type as non-xml
    correct.

    >
    > <xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns:dp="http://www.datapower.com/extensions" xmlns:func="http://exslt.org/functions" extension-element-prefixes="dp">
    > <xsl:output omit-xml-declaration="yes"/>
    ...
    ...
    > <xsl:value-of select="1"/>
    > <xsl:value-of select="$XMLByteArrayLength"/>
    > <xsl:value-of select="0"/>
    > <xsl:value-of select="$XMLByteArray"/>
    > </xsl:template>
    > </xsl:stylesheet>
    >
    > with above config AS400 application is throwing some internal error as it is unable to process the message sent from DataPower.
    > Wondering about what went wrong with above xslt. Any help is welcome.
    >
    Either I or you does not understand your specification document correctly.

    The 4th part of a request by your spec is a "XML Byte Array", the length of which has been specified in 3rd entry "Document Byte Array Length".

    You have to send "bytes" (octets), and not a sequence of decimal numbers like you did if I am right.
    Then you do not need the conversion you asked for at all.
    You should just do <xsl:copy-of select="/" /> and be fine ...

     
    Hermann<myXsltBlog/>
    Thanks Hermann for the response. I certainly didn't understand the AS/400 requirements.

    Below is the updated xsl updated per your comments and the AS/400 app still not able to process the message from DP. Is there anything wrong with the xsl code in bold per the above posted AS/400 Integration spec. Attached java client <<TestIP.java>> is working fine with no issues.

    <xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns:dp="http://www.datapower.com/extensions" extension-element-prefixes="dp" exclude-result-prefixes="dp">
    <xsl:output omit-xml-declaration="yes"/>
    <xsl:template match="/">
    <xsl:variable name="XMLByteArrayLength">
    <xsl:value-of select="dp:variable('var://service/mpgw/request-size')"/>
    </xsl:variable>
    <xsl:message dp:priority="debug">XMLByteArrayLength: <xsl:value-of select="$XMLByteArrayLength"/>
    </xsl:message>
    <xsl:value-of select="1"/>
    <xsl:value-of select="$XMLByteArrayLength"/>
    <xsl:value-of select="0"/>
    <xsl:copy-of select="/"/>
    </xsl:template>
    </xsl:stylesheet>

    Thanks in advance again.
  • HermannSW
    HermannSW
    4654 Posts

    Re: Conversion of XML file to a comma separated ASCII file

    ‏2011-08-14T01:29:16Z  
    Thanks Hermann for the response. I certainly didn't understand the AS/400 requirements.

    Below is the updated xsl updated per your comments and the AS/400 app still not able to process the message from DP. Is there anything wrong with the xsl code in bold per the above posted AS/400 Integration spec. Attached java client <<TestIP.java>> is working fine with no issues.

    <xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns:dp="http://www.datapower.com/extensions" extension-element-prefixes="dp" exclude-result-prefixes="dp">
    <xsl:output omit-xml-declaration="yes"/>
    <xsl:template match="/">
    <xsl:variable name="XMLByteArrayLength">
    <xsl:value-of select="dp:variable('var://service/mpgw/request-size')"/>
    </xsl:variable>
    <xsl:message dp:priority="debug">XMLByteArrayLength: <xsl:value-of select="$XMLByteArrayLength"/>
    </xsl:message>
    <xsl:value-of select="1"/>
    <xsl:value-of select="$XMLByteArrayLength"/>
    <xsl:value-of select="0"/>
    <xsl:copy-of select="/"/>
    </xsl:template>
    </xsl:stylesheet>

    Thanks in advance again.
    Hi,

    > ...
    > Below is the updated xsl updated per your comments and the AS/400 app
    > still not able to process the message from DP. Is there anything wrong
    > with the xsl code in bold per the above posted AS/400 Integration spec.
    > Attached java client <<TestIP.java>> is working fine with no issues.
    > ...
    your Java client is not matching your spec!

    This little stylesheet does what your Java client does (just copy the read XML file):
    <xsl:stylesheet version="1.0"
      xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
    >
      <xsl:template match="/">
        <xsl:copy-of select="."/>
      </xsl:template>
      
    </xsl:stylesheet>
    


    Please give it a try and let us know.

     
    Hermann <myXsltBlog/>
    Updated on 2014-03-25T03:13:15Z at 2014-03-25T03:13:15Z by iron-man
  • charanchowdary
    charanchowdary
    40 Posts

    Re: Conversion of XML file to a comma separated ASCII file

    ‏2011-08-14T04:35:40Z  
    • HermannSW
    • ‏2011-08-14T01:29:16Z
    Hi,

    > ...
    > Below is the updated xsl updated per your comments and the AS/400 app
    > still not able to process the message from DP. Is there anything wrong
    > with the xsl code in bold per the above posted AS/400 Integration spec.
    > Attached java client <<TestIP.java>> is working fine with no issues.
    > ...
    your Java client is not matching your spec!

    This little stylesheet does what your Java client does (just copy the read XML file):
    <pre class="java dw" data-editor-lang="java" data-pbcklang="java" dir="ltr"><xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform" > <xsl:template match="/"> <xsl:copy-of select="."/> </xsl:template> </xsl:stylesheet> </pre>

    Please give it a try and let us know.

     
    Hermann <myXsltBlog/>
    Thank you for the response.

    Tried the xsl (xml copy) posted above and AS/400 didn't like it and threw the same error. Please find the attached document which has TCPMon screenshots of both Java and DataPower (with above posted xsl to simply copy input xml file) transactions with same AS/400 app as target.

    Java Clinet --> AS/400 transaction:

    Java client is sending request xml prefixing 12 Bytes of meta data per above posted AS/400 Integration spec( you can see 12 Junk characters in TCPMon) and AS/400 is returning the desired response xml prefixing 12 Bytes of meta data per above posted AS/400 Integration spec( you can see 12 Junk characters in TCPMon).

    DataPower Clinet --> AS/400 transaction:

    DataPower is sending request xml and AS/400 is returning the error response xml prefixing 12 Bytes of meta data per above posted AS/400 Integration spec( you can see 12 Junk characters in TCPMon).

    Per my understanding, the above posted xsl(copy xml) should be updated to output metadata (per AS/400 Integration spec
    ) which should be equivalent of below 3 lines of java client code.

    out.writeInt( 1 );
    out.writeInt( xml.length ) ;
    out.writeInt( 0 ) ;

    Thanks in advance again...
  • HermannSW
    HermannSW
    4654 Posts

    Re: Conversion of XML file to a comma separated ASCII file

    ‏2011-08-14T14:42:29Z  
    Thank you for the response.

    Tried the xsl (xml copy) posted above and AS/400 didn't like it and threw the same error. Please find the attached document which has TCPMon screenshots of both Java and DataPower (with above posted xsl to simply copy input xml file) transactions with same AS/400 app as target.

    Java Clinet --> AS/400 transaction:

    Java client is sending request xml prefixing 12 Bytes of meta data per above posted AS/400 Integration spec( you can see 12 Junk characters in TCPMon) and AS/400 is returning the desired response xml prefixing 12 Bytes of meta data per above posted AS/400 Integration spec( you can see 12 Junk characters in TCPMon).

    DataPower Clinet --> AS/400 transaction:

    DataPower is sending request xml and AS/400 is returning the error response xml prefixing 12 Bytes of meta data per above posted AS/400 Integration spec( you can see 12 Junk characters in TCPMon).

    Per my understanding, the above posted xsl(copy xml) should be updated to output metadata (per AS/400 Integration spec
    ) which should be equivalent of below 3 lines of java client code.

    out.writeInt( 1 );
    out.writeInt( xml.length ) ;
    out.writeInt( 0 ) ;

    Thanks in advance again...
    HI,

    > ...
    > Java client is sending request xml prefixing 12 Bytes of meta data per above posted AS/400 Integration spec( you can see 12 Junk characters in TCPMon) and AS/400 is returning the desired response xml prefixing 12 Bytes of meta data per above posted AS/400 Integration spec( you can see 12 Junk characters in TCPMon).
    >
    > DataPower Clinet --> AS/400 transaction:
    >
    > DataPower is sending request xml and AS/400 is returning the error response xml prefixing 12 Bytes of meta data per above posted AS/400 Integration spec( you can see 12 Junk characters in TCPMon).
    >
    > Per my understanding, the above posted xsl(copy xml) should be updated to output metadata (per AS/400 Integration spec
    > ) which should be equivalent of below 3 lines of java client code.
    >
    > out.writeInt( 1 );
    > out.writeInt( xml.length ) ;
    > out.writeInt( 0 ) ;
    >
    tcpmon showed that your spec was completely correct (I misread what your Java code does).

    You cannot send values like 0x00 bytes from within a stylesheet.
    You do need binary transformations to do so.

    You may either use WTX.
    Or below solution based on the FFD shipped with the DataPower box (store:///pkcs7-convert-input.ffd).
    See posting DataPower "binary" node type (binaryNode) for details on below solution:
    https://www.ibm.com/developerworks/forums/thread.jspa?messageID=14572518#14572518

    Here you can see stylesheet "AS400.xsl" in action:
    
    $ echo 
    "<a>258</a>" | coproc2 AS400.xsl - http:
    //dp3-l3:2223 | od -Ax -tcx1 000000  \0  \0  \0 001  \0  \0  \0  \n  \0  \0  \0  \0   <a>   2 00 00 00 01 00 00 00 0a 00 00 00 00 3c 61 3e 32 000010   5   8   <   / a   > 35 38 3c 2f 61 3e 000016 $
    


    And this is your stylesheet (also attached for download), thanks for this nice task:
    
    $ cat AS400.xsl <!-- Convert input XML to AS/400 data stream: 00 00 00 01  32bitlen  00 00 00 00  XMLdata --> <xsl:stylesheet version=
    "1.0" xmlns:xsl=
    "http://www.w3.org/1999/XSL/Transform" xmlns:func=
    "http://exslt.org/functions" xmlns:dp=
    "http://www.datapower.com/extensions" extension-element-prefixes=
    "dp" > <dp:output-mapping href=
    "store:///pkcs7-convert-input.ffd" type=
    "ffd"/> <!-- This FFD converts the XML tree like 
    
    this to binary output:   <object> <message>***binary data***</message> </object> -->   <xsl:output omit-xml-declaration=
    "yes" />   <!-- convert unsigned number to base64 encoded 32bit integer.   258 --> AAABAg==  (= 00 00 01 02) --> <func:function name=
    "func:uint2b64"> <xsl:param name=
    "u"/> <func:result select=
    "dp:substring-base64( dp:radix-convert($u+4294967296,10,64), 2 )
    "/> </func:function>     <xsl:template match=
    "/"> <!-- Serialize XML input --> <xsl:variable name=
    "serialized"> <dp:serialize select=
    "." omit-xml-decl=
    "yes"/> </xsl:variable>   <!-- determine AS/400 encoding --> <xsl:variable name=
    "b64" select=
    "dp:concat-base64( dp:concat-base64( func:uint2b64(1), func:uint2b64(string-length($serialized)) ), dp:concat-base64( func:uint2b64(0), dp:encode($serialized,
    'base-64') ) )
    " />   <!-- prepare binaryNode 
    
    return 
    
    for dp:output-mapping --> <object> <message> <xsl:copy-of select=
    "dp:binary-decode($b64)" /> </message> </object> </xsl:template> </xsl:stylesheet> $
    


     
    Hermann<myXsltBlog/>
  • charanchowdary
    charanchowdary
    40 Posts

    Re: Conversion of XML file to a comma separated ASCII file

    ‏2011-08-14T22:29:45Z  
    • HermannSW
    • ‏2011-08-14T14:42:29Z
    HI,

    > ...
    > Java client is sending request xml prefixing 12 Bytes of meta data per above posted AS/400 Integration spec( you can see 12 Junk characters in TCPMon) and AS/400 is returning the desired response xml prefixing 12 Bytes of meta data per above posted AS/400 Integration spec( you can see 12 Junk characters in TCPMon).
    >
    > DataPower Clinet --> AS/400 transaction:
    >
    > DataPower is sending request xml and AS/400 is returning the error response xml prefixing 12 Bytes of meta data per above posted AS/400 Integration spec( you can see 12 Junk characters in TCPMon).
    >
    > Per my understanding, the above posted xsl(copy xml) should be updated to output metadata (per AS/400 Integration spec
    > ) which should be equivalent of below 3 lines of java client code.
    >
    > out.writeInt( 1 );
    > out.writeInt( xml.length ) ;
    > out.writeInt( 0 ) ;
    >
    tcpmon showed that your spec was completely correct (I misread what your Java code does).

    You cannot send values like 0x00 bytes from within a stylesheet.
    You do need binary transformations to do so.

    You may either use WTX.
    Or below solution based on the FFD shipped with the DataPower box (store:///pkcs7-convert-input.ffd).
    See posting DataPower "binary" node type (binaryNode) for details on below solution:
    https://www.ibm.com/developerworks/forums/thread.jspa?messageID=14572518#14572518

    Here you can see stylesheet "AS400.xsl" in action:
    <pre class="jive-pre"> $ echo "<a>258</a>" | coproc2 AS400.xsl - http: //dp3-l3:2223 | od -Ax -tcx1 000000 \0 \0 \0 001 \0 \0 \0 \n \0 \0 \0 \0 <a> 2 00 00 00 01 00 00 00 0a 00 00 00 00 3c 61 3e 32 000010 5 8 < / a > 35 38 3c 2f 61 3e 000016 $ </pre>

    And this is your stylesheet (also attached for download), thanks for this nice task:
    <pre class="jive-pre"> $ cat AS400.xsl <!-- Convert input XML to AS/400 data stream: 00 00 00 01 32bitlen 00 00 00 00 XMLdata --> <xsl:stylesheet version= "1.0" xmlns:xsl= "http://www.w3.org/1999/XSL/Transform" xmlns:func= "http://exslt.org/functions" xmlns:dp= "http://www.datapower.com/extensions" extension-element-prefixes= "dp" > <dp:output-mapping href= "store:///pkcs7-convert-input.ffd" type= "ffd"/> <!-- This FFD converts the XML tree like this to binary output: <object> <message>***binary data***</message> </object> --> <xsl:output omit-xml-declaration= "yes" /> <!-- convert unsigned number to base64 encoded 32bit integer. 258 --> AAABAg== (= 00 00 01 02) --> <func:function name= "func:uint2b64"> <xsl:param name= "u"/> <func:result select= "dp:substring-base64( dp:radix-convert($u+4294967296,10,64), 2 ) "/> </func:function> <xsl:template match= "/"> <!-- Serialize XML input --> <xsl:variable name= "serialized"> <dp:serialize select= "." omit-xml-decl= "yes"/> </xsl:variable> <!-- determine AS/400 encoding --> <xsl:variable name= "b64" select= "dp:concat-base64( dp:concat-base64( func:uint2b64(1), func:uint2b64(string-length($serialized)) ), dp:concat-base64( func:uint2b64(0), dp:encode($serialized, 'base-64') ) ) " /> <!-- prepare binaryNode return for dp:output-mapping --> <object> <message> <xsl:copy-of select= "dp:binary-decode($b64)" /> </message> </object> </xsl:template> </xsl:stylesheet> $ </pre>

     
    Hermann<myXsltBlog/>
    Thank you Hermann. The above posted AS400.xsl worked like a charm and the AS400 app is now returning the desired response with a prefix of 12 Bytes of metadata(per integration spec) which now I need to convert it to pure xml part of response rule processing. Could you share the steps to do the same. I think Response type of MPG object should be Non-XML.

    Thanks in advance again!!
  • HermannSW
    HermannSW
    4654 Posts

    Re: Conversion of XML file to a comma separated ASCII file

    ‏2011-08-15T18:07:10Z  
    Thank you Hermann. The above posted AS400.xsl worked like a charm and the AS400 app is now returning the desired response with a prefix of 12 Bytes of metadata(per integration spec) which now I need to convert it to pure xml part of response rule processing. Could you share the steps to do the same. I think Response type of MPG object should be Non-XML.

    Thanks in advance again!!
    Hi,

    > Thank you Hermann. The above posted AS400.xsl worked like a charm
    cool.

    > and the AS400 app is now returning the desired response with a prefix of 12 Bytes of metadata(per integration spec)
    > which now I need to convert it to pure xml part of response rule processing.
    > Could you share the steps to do the same.
    See below.

    > I think Response type of MPG object should be Non-XML.
    >
    Correct.

    I captured the output of AS400.xsl for the sample input from previous posting ("<a>258</a>") in AS400.dat.
    Then I created 4 testcases failing the implemented validation in new stylesheet 004SA.xsl.
    
    $ od -Ax -tx1 AS400.dat 000000 00 00 00 01 00 00 00 0a 00 00 00 00 3c 61 3e 32 000010 35 38 3c 2f 61 3e 000016 $ od -Ax -tx1 AS400.1.dat 000000 00 00 00 02 00 00 00 0a 00 00 00 00 3c 61 3e 32 000010 35 38 3c 2f 61 3e 000016 $ od -Ax -tx1 AS400.2.dat 000000 00 00 00 01 00 00 00 0a 00 00 00 00 3c 61 3e 32 000010 35 fc 3c 2f 61 3e 000016 $ od -Ax -tx1 AS400.3.dat 000000 00 00 00 01 00 00 00 09 00 00 00 00 3c 61 3e 32 000010 35 38 3c 2f 61 3e 000016 $ od -Ax -tx1 AS400.4.dat 000000 00 00 00 01 00 00 00 0a 00 00 00 01 3c 61 3e 32 000010 35 38 3c 2f 61 3e 000016 $ od -Ax -tx1 AS400.5.dat 000000 00 00 00 01 00 00 00 0a 00 00 00 00 3c 61 3e 32 000010 35 38 3c 20 61 3e 000016 $
    


    These are the error messages logged for input files
    • AS400.1.dat: xsl:message terminate=yes value='1st integer not 1'
    • AS400.2.dat: Valid base64 passed to dp:decode resulted in non UTF8: 'PGE+MjX8PC9hPg=='
    • AS400.3.dat: xsl:message terminate=yes value='XML length does not match'
    • AS400.4.dat: xsl:message terminate=yes value='3rd integer not 0'
    • AS400.5.dat: xsl:message terminate=yes value='Non-XML payload'

    Of course you can skip the validation steps 1, 3, 4 and 5 in stylesheet 004SA.xsl (also attached) if you trust getting correct XML messages from backend:
    
    <!-- Extract XMLdata from AS/400 data stream: 00 00 00 01  32bitlen  00 00 00 00  XMLdata --> <xsl:stylesheet version=
    "1.0" xmlns:xsl=
    "http://www.w3.org/1999/XSL/Transform" xmlns:func=
    "http://exslt.org/functions" xmlns:dp=
    "http://www.datapower.com/extensions" extension-element-prefixes=
    "dp" > <dp:input-mapping href=
    "store:///pkcs7-convert-input.ffd" type=
    "ffd"/> <!-- This FFD converts the input into an XML tree like 
    
    this:   <object> <message>***binary data***</message> </object> -->   <xsl:output omit-xml-declaration=
    "yes" />   <xsl:template match=
    "/"> <!-- complete input as base64 encoded string --> <xsl:variable name=
    "all" select=
    "dp:binary-encode(/object/message)" />   <!-- 1 --> <xsl:
    
    if test=
    "dp:radix-convert(dp:substring-base64($all,1,4),64,10)!=1"> <xsl:message terminate=
    "yes">1st integer not 1</xsl:message> </xsl:if>   <!-- 2 --> <!-- serialized XML input, inclusive UTF-8 validation --> <xsl:variable name=
    "serialized" select=
    "dp:decode(dp:substring-base64($all,13),'base-64')" />   <!-- 3 --> <xsl:
    
    if test=
    "dp:radix-convert(dp:substring-base64($all,5,4),64,10) != string-length($serialized)
    "> <xsl:message terminate=
    "yes">XML length does not match</xsl:message> </xsl:if>   <!-- 4 --> <xsl:
    
    if test=
    "dp:radix-convert(dp:substring-base64($all,9,4),64,10)!=0"> <xsl:message terminate=
    "yes">3rd integer not 0</xsl:message> </xsl:if>   <xsl:variable name=
    "xml"> <xsl:copy-of select=
    "dp:parse($serialized)" /> </xsl:variable>   <!-- 5 --> <xsl:
    
    if test=
    "not($xml/*)"> <xsl:message terminate=
    "yes">Non-XML payload</xsl:message> </xsl:if>   <xsl:copy-of select=
    "$xml"/> </xsl:template> </xsl:stylesheet>
    

    coproc2 output for the five testcases:
    
    $ coproc2 004SA.xsl AS400.dat http:
    //dp3-l3:2223/nonxml -s; echo <a>258</a> $ coproc2 004SA.xsl AS400.1.dat http:
    //dp3-l3:2223/nonxml -s; echo <?xml version=
    "1.0" encoding=
    "UTF-8"?> <env:Envelope xmlns:env=
    "http://schemas.xmlsoap.org/soap/envelope/"><env:Body><env:Fault><faultcode>env:Client</faultcode><faultstring>Internal Error (from client)</faultstring></env:Fault></env:Body></env:Envelope> $ coproc2 004SA.xsl AS400.2.dat http:
    //dp3-l3:2223/nonxml -s; echo <?xml version=
    "1.0" encoding=
    "UTF-8"?> <env:Envelope xmlns:env=
    "http://schemas.xmlsoap.org/soap/envelope/"><env:Body><env:Fault><faultcode>env:Client</faultcode><faultstring>Internal Error (from client)</faultstring></env:Fault></env:Body></env:Envelope> $ coproc2 004SA.xsl AS400.3.dat http:
    //dp3-l3:2223/nonxml -s; echo <?xml version=
    "1.0" encoding=
    "UTF-8"?> <env:Envelope xmlns:env=
    "http://schemas.xmlsoap.org/soap/envelope/"><env:Body><env:Fault><faultcode>env:Client</faultcode><faultstring>Internal Error (from client)</faultstring></env:Fault></env:Body></env:Envelope> $ coproc2 004SA.xsl AS400.4.dat http:
    //dp3-l3:2223/nonxml -s; echo <?xml version=
    "1.0" encoding=
    "UTF-8"?> <env:Envelope xmlns:env=
    "http://schemas.xmlsoap.org/soap/envelope/"><env:Body><env:Fault><faultcode>env:Client</faultcode><faultstring>Internal Error (from client)</faultstring></env:Fault></env:Body></env:Envelope> $ coproc2 004SA.xsl AS400.5.dat http:
    //dp3-l3:2223/nonxml -s; echo <?xml version=
    "1.0" encoding=
    "UTF-8"?> <env:Envelope xmlns:env=
    "http://schemas.xmlsoap.org/soap/envelope/"><env:Body><env:Fault><faultcode>env:Client</faultcode><faultstring>Internal Error (from client)</faultstring></env:Fault></env:Body></env:Envelope> $
    


     
    Hermann<myXsltBlog/>
  • charanchowdary
    charanchowdary
    40 Posts

    Re: Conversion of XML file to a comma separated ASCII file

    ‏2011-08-15T21:08:12Z  
    • HermannSW
    • ‏2011-08-15T18:07:10Z
    Hi,

    > Thank you Hermann. The above posted AS400.xsl worked like a charm
    cool.

    > and the AS400 app is now returning the desired response with a prefix of 12 Bytes of metadata(per integration spec)
    > which now I need to convert it to pure xml part of response rule processing.
    > Could you share the steps to do the same.
    See below.

    > I think Response type of MPG object should be Non-XML.
    >
    Correct.

    I captured the output of AS400.xsl for the sample input from previous posting ("<a>258</a>") in AS400.dat.
    Then I created 4 testcases failing the implemented validation in new stylesheet 004SA.xsl.
    <pre class="jive-pre"> $ od -Ax -tx1 AS400.dat 000000 00 00 00 01 00 00 00 0a 00 00 00 00 3c 61 3e 32 000010 35 38 3c 2f 61 3e 000016 $ od -Ax -tx1 AS400.1.dat 000000 00 00 00 02 00 00 00 0a 00 00 00 00 3c 61 3e 32 000010 35 38 3c 2f 61 3e 000016 $ od -Ax -tx1 AS400.2.dat 000000 00 00 00 01 00 00 00 0a 00 00 00 00 3c 61 3e 32 000010 35 fc 3c 2f 61 3e 000016 $ od -Ax -tx1 AS400.3.dat 000000 00 00 00 01 00 00 00 09 00 00 00 00 3c 61 3e 32 000010 35 38 3c 2f 61 3e 000016 $ od -Ax -tx1 AS400.4.dat 000000 00 00 00 01 00 00 00 0a 00 00 00 01 3c 61 3e 32 000010 35 38 3c 2f 61 3e 000016 $ od -Ax -tx1 AS400.5.dat 000000 00 00 00 01 00 00 00 0a 00 00 00 00 3c 61 3e 32 000010 35 38 3c 20 61 3e 000016 $ </pre>

    These are the error messages logged for input files
    • AS400.1.dat: xsl:message terminate=yes value='1st integer not 1'
    • AS400.2.dat: Valid base64 passed to dp:decode resulted in non UTF8: 'PGE+MjX8PC9hPg=='
    • AS400.3.dat: xsl:message terminate=yes value='XML length does not match'
    • AS400.4.dat: xsl:message terminate=yes value='3rd integer not 0'
    • AS400.5.dat: xsl:message terminate=yes value='Non-XML payload'

    Of course you can skip the validation steps 1, 3, 4 and 5 in stylesheet 004SA.xsl (also attached) if you trust getting correct XML messages from backend:
    <pre class="jive-pre"> <!-- Extract XMLdata from AS/400 data stream: 00 00 00 01 32bitlen 00 00 00 00 XMLdata --> <xsl:stylesheet version= "1.0" xmlns:xsl= "http://www.w3.org/1999/XSL/Transform" xmlns:func= "http://exslt.org/functions" xmlns:dp= "http://www.datapower.com/extensions" extension-element-prefixes= "dp" > <dp:input-mapping href= "store:///pkcs7-convert-input.ffd" type= "ffd"/> <!-- This FFD converts the input into an XML tree like this: <object> <message>***binary data***</message> </object> --> <xsl:output omit-xml-declaration= "yes" /> <xsl:template match= "/"> <!-- complete input as base64 encoded string --> <xsl:variable name= "all" select= "dp:binary-encode(/object/message)" /> <!-- 1 --> <xsl: if test= "dp:radix-convert(dp:substring-base64($all,1,4),64,10)!=1"> <xsl:message terminate= "yes">1st integer not 1</xsl:message> </xsl:if> <!-- 2 --> <!-- serialized XML input, inclusive UTF-8 validation --> <xsl:variable name= "serialized" select= "dp:decode(dp:substring-base64($all,13),'base-64')" /> <!-- 3 --> <xsl: if test= "dp:radix-convert(dp:substring-base64($all,5,4),64,10) != string-length($serialized) "> <xsl:message terminate= "yes">XML length does not match</xsl:message> </xsl:if> <!-- 4 --> <xsl: if test= "dp:radix-convert(dp:substring-base64($all,9,4),64,10)!=0"> <xsl:message terminate= "yes">3rd integer not 0</xsl:message> </xsl:if> <xsl:variable name= "xml"> <xsl:copy-of select= "dp:parse($serialized)" /> </xsl:variable> <!-- 5 --> <xsl: if test= "not($xml/*)"> <xsl:message terminate= "yes">Non-XML payload</xsl:message> </xsl:if> <xsl:copy-of select= "$xml"/> </xsl:template> </xsl:stylesheet> </pre>
    coproc2 output for the five testcases:
    <pre class="jive-pre"> $ coproc2 004SA.xsl AS400.dat http: //dp3-l3:2223/nonxml -s; echo <a>258</a> $ coproc2 004SA.xsl AS400.1.dat http: //dp3-l3:2223/nonxml -s; echo <?xml version= "1.0" encoding= "UTF-8"?> <env:Envelope xmlns:env= "http://schemas.xmlsoap.org/soap/envelope/"><env:Body><env:Fault><faultcode>env:Client</faultcode><faultstring>Internal Error (from client)</faultstring></env:Fault></env:Body></env:Envelope> $ coproc2 004SA.xsl AS400.2.dat http: //dp3-l3:2223/nonxml -s; echo <?xml version= "1.0" encoding= "UTF-8"?> <env:Envelope xmlns:env= "http://schemas.xmlsoap.org/soap/envelope/"><env:Body><env:Fault><faultcode>env:Client</faultcode><faultstring>Internal Error (from client)</faultstring></env:Fault></env:Body></env:Envelope> $ coproc2 004SA.xsl AS400.3.dat http: //dp3-l3:2223/nonxml -s; echo <?xml version= "1.0" encoding= "UTF-8"?> <env:Envelope xmlns:env= "http://schemas.xmlsoap.org/soap/envelope/"><env:Body><env:Fault><faultcode>env:Client</faultcode><faultstring>Internal Error (from client)</faultstring></env:Fault></env:Body></env:Envelope> $ coproc2 004SA.xsl AS400.4.dat http: //dp3-l3:2223/nonxml -s; echo <?xml version= "1.0" encoding= "UTF-8"?> <env:Envelope xmlns:env= "http://schemas.xmlsoap.org/soap/envelope/"><env:Body><env:Fault><faultcode>env:Client</faultcode><faultstring>Internal Error (from client)</faultstring></env:Fault></env:Body></env:Envelope> $ coproc2 004SA.xsl AS400.5.dat http: //dp3-l3:2223/nonxml -s; echo <?xml version= "1.0" encoding= "UTF-8"?> <env:Envelope xmlns:env= "http://schemas.xmlsoap.org/soap/envelope/"><env:Body><env:Fault><faultcode>env:Client</faultcode><faultstring>Internal Error (from client)</faultstring></env:Fault></env:Body></env:Envelope> $ </pre>

     
    Hermann<myXsltBlog/>
    Thank you so much for the quick response. Above style sheet looks good to me though we ran in to below Network error during binary response processing for this I have opened a PMR as even the simple text to XML wrapping scenario per the technote @ http://www-01.ibm.com/support/docview.wss?uid=swg21321379 is also resulting in below network error.

    10.93.19.81 0x80c00008 mpgw (test): rule (test-sp-response-rule): implied action Read input as binary data failed: Network error

    Attached <<debug-syslog-10455584.doc>> word document has the debug level sys log entries for the transaction id # 10455584 in question along with the probe screenshot which shows the received binary data from the back end AS/400 application.

    Thank you again for your assistance!!!!
  • HermannSW
    HermannSW
    4654 Posts

    Re: Conversion of XML file to a comma separated ASCII file

    ‏2011-08-15T21:24:53Z  
    Thank you so much for the quick response. Above style sheet looks good to me though we ran in to below Network error during binary response processing for this I have opened a PMR as even the simple text to XML wrapping scenario per the technote @ http://www-01.ibm.com/support/docview.wss?uid=swg21321379 is also resulting in below network error.

    10.93.19.81 0x80c00008 mpgw (test): rule (test-sp-response-rule): implied action Read input as binary data failed: Network error

    Attached <<debug-syslog-10455584.doc>> word document has the debug level sys log entries for the transaction id # 10455584 in question along with the probe screenshot which shows the received binary data from the back end AS/400 application.

    Thank you again for your assistance!!!!
    > Thank you so much for the quick response. Above style sheet looks good to me though we ran in to below
    > Network error during binary response processing for this I have opened a PMR as even the simple text to
    > XML wrapping scenario per the technote @ http://www-01.ibm.com/support/docview.wss?uid=swg21321379 is
    > also resulting in below network error.
    >
    > 10.93.19.81 0x80c00008 mpgw (test): rule (test-sp-response-rule): implied action Read input as binary data failed: Network error
    >
    > Attached <<debug-syslog-10455584.doc>> word document has the debug level sys log entries for the transaction id # 10455584 in question along with the probe screenshot which shows the received binary data from the back end AS/400 application.
    >

    Below two lines from your syslog show that DataPower is trying to parse your response message as XML.
    But your response is binary.

    What you need to have in place is:
    • Response Type "Non-XML"
    • Have stylesheet "004SA.xsl" in a "Binary Transform Action"

    The parse error seems to indicate that you have a "normal" transform action in response rule ...
    
    ... 14:23:06        xmlparse        debug   10455584        response        10.93.19.81     0x80e003a6      mpgw (test): Parsing document: 
    'http://161.159.103.117:4201/AWD-V1' 14:23:06  multistep       error   10455584        response        10.93.19.81     0x80c00008      mpgw (test): rule (test-sp-response-rule): implied action Read input as binary data failed: Network error ...
    


     
    Hermann<myXsltBlog/>
  • charanchowdary
    charanchowdary
    40 Posts

    Re: Conversion of XML file to a comma separated ASCII file

    ‏2011-08-15T21:36:50Z  
    • HermannSW
    • ‏2011-08-15T21:24:53Z
    > Thank you so much for the quick response. Above style sheet looks good to me though we ran in to below
    > Network error during binary response processing for this I have opened a PMR as even the simple text to
    > XML wrapping scenario per the technote @ http://www-01.ibm.com/support/docview.wss?uid=swg21321379 is
    > also resulting in below network error.
    >
    > 10.93.19.81 0x80c00008 mpgw (test): rule (test-sp-response-rule): implied action Read input as binary data failed: Network error
    >
    > Attached <<debug-syslog-10455584.doc>> word document has the debug level sys log entries for the transaction id # 10455584 in question along with the probe screenshot which shows the received binary data from the back end AS/400 application.
    >

    Below two lines from your syslog show that DataPower is trying to parse your response message as XML.
    But your response is binary.

    What you need to have in place is:
    • Response Type "Non-XML"
    • Have stylesheet "004SA.xsl" in a "Binary Transform Action"

    The parse error seems to indicate that you have a "normal" transform action in response rule ...
    <pre class="jive-pre"> ... 14:23:06 xmlparse debug 10455584 response 10.93.19.81 0x80e003a6 mpgw (test): Parsing document: 'http://161.159.103.117:4201/AWD-V1' 14:23:06 multistep error 10455584 response 10.93.19.81 0x80c00008 mpgw (test): rule (test-sp-response-rule): implied action Read input as binary data failed: Network error ... </pre>

     
    Hermann<myXsltBlog/>
    > What you need to have in place is:
    > * Response Type "Non-XML"
    > * Have stylesheet "004SA.xsl" in a "Binary Transform Action"

    I actually have my Response Type as "Non-XML" and I do have the "004SA.xsl" in a "Binary Transform Action" per the attached probe export <<test-probe-export-capture.zip>> .

    Thank you again!!!.
  • HermannSW
    HermannSW
    4654 Posts

    Re: Conversion of XML file to a comma separated ASCII file

    ‏2011-08-16T09:23:35Z  
    > What you need to have in place is:
    > * Response Type "Non-XML"
    > * Have stylesheet "004SA.xsl" in a "Binary Transform Action"

    I actually have my Response Type as "Non-XML" and I do have the "004SA.xsl" in a "Binary Transform Action" per the attached probe export <<test-probe-export-capture.zip>> .

    Thank you again!!!.
    Hi,
    >
    > I actually have my Response Type as "Non-XML" and I do have the "004SA.xsl" in a "Binary Transform Action" per the attached probe export <<test-probe-export-capture.zip>> .
    >

    I could not immediately see what is wrong with your configuration.

    So I just built the attached MPGW from scratch making use of my previously posted stylesheets AS400.xsl and 004SA.xsl.

    My DataPower box dp3-l3 service listens on port 4445 and goes to backend tcp://dp4-l3:4444.

    I use netcat tool as TCP server.
    First I tested the setup.
    This is TCP server emulation on dp4-l3 returning "test" received from client:
    
    [stammw@dp4-l3 ~]$ nc -l 4444 <AS400.dat | od -tcx1 0000000   t   e   s   t  \n 74 65 73 74 0a 0000005 [stammw@dp4-l3 ~]$
    


    This is the (test) client (receiving the AS400.dat response):
    
    $ echo 
    "test" | nc dp4-l3 4444 - | od -tcx1 0000000  \0  \0  \0 001  \0  \0  \0  \n  \0  \0  \0  \0   <a>   2 00 00 00 01 00 00 00 0a 00 00 00 00 3c 61 3e 32 0000020   5   8   <   / a   > 35 38 3c 2f 61 3e 0000026 $
    


    Now after this works here is TCP server on dp4-l3 again (with request received from DataPower):
    
    [stammw@dp4-l3 ~]$ nc -l 4444 <AS400.dat | od -tcx1 0000000  \0  \0  \0 001  \0  \0  \0  \n  \0  \0  \0  \0   <   b   >   1 00 00 00 01 00 00 00 0a 00 00 00 00 3c 62 3e 31 0000020   2   3   <   / b   > 32 33 3c 2f 62 3e 0000026 [stammw@dp4-l3 ~]$
    


    And this is the curl client going through DataPower service to TCP server receiving the
    decoded AS400.dat XML message (different to the message sent!):
    
    $ echo 
    "<b>123</b>" | curl --data-binary @- http:
    //dp3-l3:4445; echo <a>258</a> $
    


    Now you have to look what is wrong in your configuration ...

     
    Hermann<myXsltBlog/>

    Attachments

  • charanchowdary
    charanchowdary
    40 Posts

    Re: Conversion of XML file to a comma separated ASCII file

    ‏2011-08-16T15:41:04Z  
    • HermannSW
    • ‏2011-08-16T09:23:35Z
    Hi,
    >
    > I actually have my Response Type as "Non-XML" and I do have the "004SA.xsl" in a "Binary Transform Action" per the attached probe export <<test-probe-export-capture.zip>> .
    >

    I could not immediately see what is wrong with your configuration.

    So I just built the attached MPGW from scratch making use of my previously posted stylesheets AS400.xsl and 004SA.xsl.

    My DataPower box dp3-l3 service listens on port 4445 and goes to backend tcp://dp4-l3:4444.

    I use netcat tool as TCP server.
    First I tested the setup.
    This is TCP server emulation on dp4-l3 returning "test" received from client:
    <pre class="jive-pre"> [stammw@dp4-l3 ~]$ nc -l 4444 <AS400.dat | od -tcx1 0000000 t e s t \n 74 65 73 74 0a 0000005 [stammw@dp4-l3 ~]$ </pre>

    This is the (test) client (receiving the AS400.dat response):
    <pre class="jive-pre"> $ echo "test" | nc dp4-l3 4444 - | od -tcx1 0000000 \0 \0 \0 001 \0 \0 \0 \n \0 \0 \0 \0 <a> 2 00 00 00 01 00 00 00 0a 00 00 00 00 3c 61 3e 32 0000020 5 8 < / a > 35 38 3c 2f 61 3e 0000026 $ </pre>

    Now after this works here is TCP server on dp4-l3 again (with request received from DataPower):
    <pre class="jive-pre"> [stammw@dp4-l3 ~]$ nc -l 4444 <AS400.dat | od -tcx1 0000000 \0 \0 \0 001 \0 \0 \0 \n \0 \0 \0 \0 < b > 1 00 00 00 01 00 00 00 0a 00 00 00 00 3c 62 3e 31 0000020 2 3 < / b > 32 33 3c 2f 62 3e 0000026 [stammw@dp4-l3 ~]$ </pre>

    And this is the curl client going through DataPower service to TCP server receiving the
    decoded AS400.dat XML message (different to the message sent!):
    <pre class="jive-pre"> $ echo "<b>123</b>" | curl --data-binary @- http: //dp3-l3:4445; echo <a>258</a> $ </pre>

    Now you have to look what is wrong in your configuration ...

     
    Hermann<myXsltBlog/>
    1. I verified both of our configs are similar and they both are looking good.
    2. a) I did import your MPGW(4444)to my XI50-->9235-4BX device running on 3.8.2.3 (I don't think this should have any compatibility issues as source 3.8.2.5 is on bit higher level than target 3.8.2.3)
    b)Only changed the back side URL to point it to AS/400 application.

    and when tested I see the same network error that I see with my MPGW. Below are the service variable values related to the error on step 0 of the response leg. Attaching the probe Export Capture <<4444-probe-ExportCapture.zip>>.

    var://service/error-code string 0x00530001
    var://service/error-message string 'Network error'
    var://service/error-subcode string 0x00530001

    3. PMR support team is analyzing the pcap files currently.

    Thank you again!!!
  • HermannSW
    HermannSW
    4654 Posts

    Re: Conversion of XML file to a comma separated ASCII file

    ‏2011-08-16T17:50:29Z  
    1. I verified both of our configs are similar and they both are looking good.
    2. a) I did import your MPGW(4444)to my XI50-->9235-4BX device running on 3.8.2.3 (I don't think this should have any compatibility issues as source 3.8.2.5 is on bit higher level than target 3.8.2.3)
    b)Only changed the back side URL to point it to AS/400 application.

    and when tested I see the same network error that I see with my MPGW. Below are the service variable values related to the error on step 0 of the response leg. Attaching the probe Export Capture <<4444-probe-ExportCapture.zip>>.

    var://service/error-code string 0x00530001
    var://service/error-message string 'Network error'
    var://service/error-subcode string 0x00530001

    3. PMR support team is analyzing the pcap files currently.

    Thank you again!!!
    Hi,

    the Probe isn't helpful here.

    If you want to do next step of investigation you should
    • setup the config against a AS/400 emulating netcat server as I did (that will work as I showed)
    • setup as you already have against real AS/400 server.

    For both scenarios you should take a DataPower packet capture on the interface going to AS/400 (emulation or real server).

    Then you will be able to identify the difference by (Wireshark) "Follow TCP stream" in both pcaps.
    Since AS/400 emulation works (as shown above) we already know what data comes back to DataPower.
    Your real AS/400 system seems to do something different (not neccesarily data, but FINs or ACKs).

     
    Hermann<myXsltBlog/>