Topic
  • 21 replies
  • Latest Post - ‏2011-11-11T11:24:37Z by Jaango
HermannSW
HermannSW
4901 Posts

Pinned topic xpath tool

‏2010-04-12T14:28:53Z |
It is quite useful to be able to apply XPath statements to a (probably unknown) XML file.

Below stylsheet xpath.xsl and shell script xpath (for Linux or cygwin) together build a little xpath tool.

The only (minor) drawback is that the added comment for passing the xpath is part of the modified XML document.
But comments are not that important normally.


$ xpath 
"count(//*)" x.xml 4 $ xpath 
"string(//*[@attr]/@attr)" x.xml tst $ xpath 
"count(//comment())" x.xml 1 $ cat x.xml <?xml version=
"1.0" encoding=
"UTF-8"?> <a>
<b>2</b>3<c attr=
"tst">4<d/>5</c>6</a> $


Applying to itself:

$ xpath 
"string(//*[name()='xsl:copy-of']/@select)" xpath.xsl dyn:evaluate(string(
//comment()[last()])) $


Shell script xpath:

#!/bin/bash tempfile=/tmp/xpath.input.xml   cp $2 $tempfile echo 
"<!--" >> $tempfile echo 
"$1" >>$tempfile echo 
"-->" >> $tempfile   #xsltproc xpath.xsl $tempfile curl --data-binary @$tempfile http:
//dp1-l3.boeblingen.de.ibm.com:2048/xpath   rm $tempfile


Stylesheet xpath.xsl (locally for xsltproc, in xform action on box, otherwise):

$ cat xpath.xsl <xsl:stylesheet version=
"1.0" xmlns:xsl=
"http://www.w3.org/1999/XSL/Transform" xmlns:dyn=
"http://exslt.org/dynamic" extension-element-prefixes=
"dyn" > <xsl:output method=
"xml" omit-xml-declaration=
"yes"/> <xsl:template match=
"/"> <xsl:copy-of select=
"dyn:evaluate(string(//comment()[last()]))"/> </xsl:template>   </xsl:stylesheet> $
Updated on 2011-11-11T11:24:37Z at 2011-11-11T11:24:37Z by Jaango
  • HermannSW
    HermannSW
    4901 Posts

    Re: xpath tool

    ‏2010-04-22T08:22:49Z  
    A colleague asked why "xpath tool" does not work with xsltproc.
    In fact it did not work with DP, either, if the XML file contained at least one comment -- I had posted a broken version of xpath.xsl.

    Here is the working version:
    $ cat xpath.xsl
    <xsl:stylesheet version="1.0"
      xmlns:xsl="http://www.w3.org/1999/XSL/Transform" 
      xmlns:dyn="http://exslt.org/dynamic"
      extension-element-prefixes="dyn"
    >
      <xsl:output method="xml" omit-xml-declaration="yes"/>
      <xsl:template match="/">
        <xsl:copy-of select="dyn:evaluate(string((//comment())[last()]))"/>
      </xsl:template>
     
    </xsl:stylesheet>
    $
    


    The problem was a missing bracket pair around the //comment() statement:
    $ diff xpath.xsl xpath.xsl.old 
    8c8
    <     <xsl:copy-of select="dyn:evaluate(string((//comment())[last()]))"/>
    ---
    >     <xsl:copy-of select="dyn:evaluate(string(//comment()[last()]))"/>
    $
    
    Updated on 2014-03-25T03:44:59Z at 2014-03-25T03:44:59Z by iron-man
  • HermannSW
    HermannSW
    4901 Posts

    Re: xpath tool

    ‏2010-05-20T08:45:44Z  
    For convenience (verified on Linux and Cygwin):
    xpath tool as one-file-solution (shell script+stylesheet) with usage information.
    #!/bin/bash
    #
    # xpath tool
    #
    if [ "$2" = "" ]; then
      echo "usage: `basename $0` <XPath expression> <XML file>"
      exit
    fi
     
    tempxsl=/tmp/temp.xsl
    sed -n "/<\!--delim-->/,/<\!--delim-->/p" $0 >$tempxsl
     
    PROC="xsltproc $tempxsl -"
    #PROC="curl --data-binary @- http://dp1-l3.boeblingen.de.ibm.com:2048/xpath"
     
    echo "`cat $2` <"\!"-- $1 -->" | $PROC
     
    rm $tempxsl
    exit
     
    <!--delim-->
    <xsl:stylesheet version="1.0"
      xmlns:xsl="http://www.w3.org/1999/XSL/Transform" 
      xmlns:dyn="http://exslt.org/dynamic"
      extension-element-prefixes="dyn"
    >
      <xsl:output method="xml" omit-xml-declaration="yes"/>
      <xsl:template match="/">
        <xsl:copy-of select="dyn:evaluate(string((//comment())[last()]))"/>
      </xsl:template>
     
    </xsl:stylesheet>
    <!--delim-->
    
    Updated on 2014-03-25T03:43:55Z at 2014-03-25T03:43:55Z by iron-man
  • msascha
    msascha
    52 Posts

    Re: xpath tool

    ‏2010-05-20T19:24:30Z  
    • HermannSW
    • ‏2010-05-20T08:45:44Z
    For convenience (verified on Linux and Cygwin):
    xpath tool as one-file-solution (shell script+stylesheet) with usage information.
    <pre class="java dw" data-editor-lang="java" data-pbcklang="java" dir="ltr">#!/bin/bash # # xpath tool # if [ "$2" = "" ]; then echo "usage: `basename $0` <XPath expression> <XML file>" exit fi tempxsl=/tmp/temp.xsl sed -n "/<\!--delim-->/,/<\!--delim-->/p" $0 >$tempxsl PROC="xsltproc $tempxsl -" #PROC="curl --data-binary @- http://dp1-l3.boeblingen.de.ibm.com:2048/xpath" echo "`cat $2` <"\!"-- $1 -->" | $PROC rm $tempxsl exit <!--delim--> <xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns:dyn="http://exslt.org/dynamic" extension-element-prefixes="dyn" > <xsl:output method="xml" omit-xml-declaration="yes"/> <xsl:template match="/"> <xsl:copy-of select="dyn:evaluate(string((//comment())[last()]))"/> </xsl:template> </xsl:stylesheet> <!--delim--> </pre>
    Hi Herman,

    if you transmit the stylesheet/xpath as HTTP-Header (base64-encoded) you don't have to worry about comments in the source XML. :-)

    Sascha
  • HermannSW
    HermannSW
    4901 Posts

    Re: xpath tool

    ‏2010-05-22T17:40:11Z  
    • msascha
    • ‏2010-05-20T19:24:30Z
    Hi Herman,

    if you transmit the stylesheet/xpath as HTTP-Header (base64-encoded) you don't have to worry about comments in the source XML. :-)

    Sascha
    Hi Sascha,

    > if you transmit the stylesheet/xpath as HTTP-Header (base64-encoded)
    > you don't have to worry about comments in the source XML. :-)
    good catch!

    But HTTP-Headers are available to DataPower, not to xsltproc.
    On the other hand xsltproc allows to pass string parameters.

    In order to not rely on another tool for base64-encoding the xpath string
    in the DataPower case I hijacked the "Authorization" header string.
    The -u option of curl does the conversion then ...

    
    #!/bin/bash # # xpath tool # 
    
    if [ 
    "$2" = 
    "" ]; then echo 
    "usage: `basename $0` <XPath expression> <XML file>" exit fi   # 
    
    for DataPower use comment out 
    
    this line xsltproc_xsl=/tmp/temp.xsl     
    
    if [ 
    "$xsltproc_xsl" != 
    "" ]; then sed -n 
    "/<\!--delim-->/,/<\!--delim-->/p" $0 >$xsltproc_xsl   xsltproc --stringparam xpathpar 
    "$1" $xsltproc_xsl $2   rm $xsltproc_xsl 
    
    else url=http:
    //dp1-l3.boeblingen.de.ibm.com:2048/xpath   curl -u 
    "$1:" --data-binary @$2 $url; echo fi   exit     <!--delim--> <xsl:stylesheet version=
    "1.0" xmlns:xsl=
    "http://www.w3.org/1999/XSL/Transform" xmlns:dyn=
    "http://exslt.org/dynamic" xmlns:dp=
    "http://www.datapower.com/extensions" extension-element-prefixes=
    "dp dyn" > <xsl:output method=
    "xml" omit-xml-declaration=
    "yes"/>   <xsl:param name=
    "xpathpar"/>   <xsl:template match=
    "/"> <xsl:variable name=
    "xpath"> <xsl:choose> <xsl:when test=
    "function-available('dp:http-request-header')"> <xsl:variable name=
    "auth" select=
    "dp:http-request-header('Authorization')"/> <xsl:variable name=
    "b64" select=
    "substring-after($auth,'Basic ')"/> <xsl:variable name=
    "user_" select=
    "dp:binary-decode($b64)"/> <xsl:value-of select=
    "substring($user_,1,string-length($user_)-1)"/> </xsl:when> <xsl:otherwise> <xsl:value-of select=
    "$xpathpar"/> </xsl:otherwise> </xsl:choose> </xsl:variable>   <xsl:copy-of select=
    "dyn:evaluate($xpath)"/> </xsl:template>   </xsl:stylesheet> <!--delim-->
    
  • HermannSW
    HermannSW
    4901 Posts

    Re: xpath tool

    ‏2010-07-20T11:14:01Z  
    This slide contains two additional samples of xpath tool usage:
    http://www-01.ibm.com/support/docview.wss?uid=swg27019118&aid=1#page=13
  • HermannSW
    HermannSW
    4901 Posts

    Re: xpath tool

    ‏2010-07-20T16:35:00Z  
    I needed to print out the string values of an attribute for each node of a node-set.

    While "string(/*/*@name)" works it does only return the string value of the attribute
    of the first node of the node-set (http://www.w3.org/TR/xpath/#section-String-Functions).

    By "string(/*/*[2]/@name)" the string value of the second node's attribut was returned.
    But the output of all attribute names for a given node-set was only possible with a shell script.

    So I asked colleagues on how to do this better.

    The answer I got was really good:

    Yes, I did -- str:concat() accepts a node-set as argument and returns the concatenation of all node's string values.
    Find at the bottom the modified one-file-solution xpath++ which works for xsltproc and DataPower.

    For my use case the str-module only needed to be added.
    For completeness the namespace prefix declarations for all modules of EXSLT were added, see the difference here:
    
    $ diff xpath xpath++ 31a32 >   xmlns:date=
    "http://exslt.org/dates-and-times" 32a34,40 >   xmlns:exsl=
    "http://exslt.org/common" >   xmlns:func=
    "http://exslt.org/functions" >   xmlns:math=
    "http://exslt.org/math" >   xmlns:random=
    "http://exslt.org/random" >   xmlns:regexp=
    "http://exslt.org/regular-expressions" >   xmlns:set=
    "http://exslt.org/sets" >   xmlns:str=
    "http://exslt.org/strings" $
    


    Find here some samples of what can be done with xpath++.

    set:distinct
    
    $ cat data.xml <a> <b>5</b> <b>6</b> <b>5</b> </a> $ xpath++ 
    "set:distinct(/*/*)" data.xml <b>5</b><b>6</b> $ xpath++ 
    "/*/*" data.xml <b>5</b><b>6</b><b>5</b> $
    


    exsl:object-type
    
    $ xpath++ 
    "exsl:object-type(/*/*)" data.xml node-set $
    


    str:concat
    
    $ cat position.xml <board> <piece name=
    "king"   color=
    "black"><field>a8</field></piece> <piece name=
    "knight" color=
    "white"><field>a6</field></piece> <piece name=
    "king"   color=
    "white"><field>b6</field></piece> <piece name=
    "bishop" color=
    "white"><field>e4</field></piece> </board> $ xpath++ 
    "str:concat(/board/piece/@name)" position.xml kingknightkingbishop $ xpath++ 
    "str:concat(/board/piece/@name | /board/piece/@color)" position.xml kingblackknightwhitekingwhitebishopwhite $
    


    Finally here is the listing of xpath++ tool:
    
    #!/bin/bash # # xpath tool # 
    
    if [ 
    "$2" = 
    "" ]; then echo 
    "usage: `basename $0` <XPath expression> <XML file>" exit fi   # 
    
    for DataPower use comment out 
    
    this line xsltproc_xsl=/tmp/temp.xsl     
    
    if [ 
    "$xsltproc_xsl" != 
    "" ]; then sed -n 
    "/<\!--delim-->/,/<\!--delim-->/p" $0 >$xsltproc_xsl   xsltproc --stringparam xpathpar 
    "$1" $xsltproc_xsl $2   rm $xsltproc_xsl 
    
    else url=http:
    //dp1-l3.boeblingen.de.ibm.com:2048/xpath   curl -u 
    "$1:" --data-binary @$2 $url; echo fi   exit     <!--delim--> <xsl:stylesheet version=
    "1.0" xmlns:xsl=
    "http://www.w3.org/1999/XSL/Transform" xmlns:date=
    "http://exslt.org/dates-and-times" xmlns:dyn=
    "http://exslt.org/dynamic" xmlns:exsl=
    "http://exslt.org/common" xmlns:func=
    "http://exslt.org/functions" xmlns:math=
    "http://exslt.org/math" xmlns:random=
    "http://exslt.org/random" xmlns:regexp=
    "http://exslt.org/regular-expressions" xmlns:set=
    "http://exslt.org/sets" xmlns:str=
    "http://exslt.org/strings" xmlns:dp=
    "http://www.datapower.com/extensions" extension-element-prefixes=
    "dp dyn" > <xsl:output method=
    "xml" omit-xml-declaration=
    "yes"/>   <xsl:param name=
    "xpathpar" />   <xsl:template match=
    "/"> <xsl:variable name=
    "xpath"> <xsl:choose> <xsl:when test=
    "function-available('dp:http-request-header')"> <xsl:variable name=
    "auth" select=
    "dp:http-request-header('Authorization')"/> <xsl:variable name=
    "b64" select=
    "substring-after($auth,'Basic ')"/> <xsl:variable name=
    "user_" select=
    "dp:binary-decode($b64)"/> <xsl:value-of select=
    "substring($user_,1,string-length($user_)-1)"/> </xsl:when> <xsl:otherwise> <xsl:value-of select=
    "$xpathpar"/> </xsl:otherwise> </xsl:choose> </xsl:variable>   <xsl:copy-of select=
    "dyn:evaluate($xpath)"/> </xsl:template>   </xsl:stylesheet> <!--delim--> $
    
  • HermannSW
    HermannSW
    4901 Posts

    Re: xpath tool

    ‏2010-08-17T12:58:05Z  
    • HermannSW
    • ‏2010-07-20T16:35:00Z
    I needed to print out the string values of an attribute for each node of a node-set.

    While "string(/*/*@name)" works it does only return the string value of the attribute
    of the first node of the node-set (http://www.w3.org/TR/xpath/#section-String-Functions).

    By "string(/*/*[2]/@name)" the string value of the second node's attribut was returned.
    But the output of all attribute names for a given node-set was only possible with a shell script.

    So I asked colleagues on how to do this better.

    The answer I got was really good:

    Yes, I did -- str:concat() accepts a node-set as argument and returns the concatenation of all node's string values.
    Find at the bottom the modified one-file-solution xpath++ which works for xsltproc and DataPower.

    For my use case the str-module only needed to be added.
    For completeness the namespace prefix declarations for all modules of EXSLT were added, see the difference here:
    <pre class="jive-pre"> $ diff xpath xpath++ 31a32 > xmlns:date= "http://exslt.org/dates-and-times" 32a34,40 > xmlns:exsl= "http://exslt.org/common" > xmlns:func= "http://exslt.org/functions" > xmlns:math= "http://exslt.org/math" > xmlns:random= "http://exslt.org/random" > xmlns:regexp= "http://exslt.org/regular-expressions" > xmlns:set= "http://exslt.org/sets" > xmlns:str= "http://exslt.org/strings" $ </pre>

    Find here some samples of what can be done with xpath++.

    set:distinct
    <pre class="jive-pre"> $ cat data.xml <a> <b>5</b> <b>6</b> <b>5</b> </a> $ xpath++ "set:distinct(/*/*)" data.xml <b>5</b><b>6</b> $ xpath++ "/*/*" data.xml <b>5</b><b>6</b><b>5</b> $ </pre>

    exsl:object-type
    <pre class="jive-pre"> $ xpath++ "exsl:object-type(/*/*)" data.xml node-set $ </pre>

    str:concat
    <pre class="jive-pre"> $ cat position.xml <board> <piece name= "king" color= "black"><field>a8</field></piece> <piece name= "knight" color= "white"><field>a6</field></piece> <piece name= "king" color= "white"><field>b6</field></piece> <piece name= "bishop" color= "white"><field>e4</field></piece> </board> $ xpath++ "str:concat(/board/piece/@name)" position.xml kingknightkingbishop $ xpath++ "str:concat(/board/piece/@name | /board/piece/@color)" position.xml kingblackknightwhitekingwhitebishopwhite $ </pre>

    Finally here is the listing of xpath++ tool:
    <pre class="jive-pre"> #!/bin/bash # # xpath tool # if [ "$2" = "" ]; then echo "usage: `basename $0` <XPath expression> <XML file>" exit fi # for DataPower use comment out this line xsltproc_xsl=/tmp/temp.xsl if [ "$xsltproc_xsl" != "" ]; then sed -n "/<\!--delim-->/,/<\!--delim-->/p" $0 >$xsltproc_xsl xsltproc --stringparam xpathpar "$1" $xsltproc_xsl $2 rm $xsltproc_xsl else url=http: //dp1-l3.boeblingen.de.ibm.com:2048/xpath curl -u "$1:" --data-binary @$2 $url; echo fi exit <!--delim--> <xsl:stylesheet version= "1.0" xmlns:xsl= "http://www.w3.org/1999/XSL/Transform" xmlns:date= "http://exslt.org/dates-and-times" xmlns:dyn= "http://exslt.org/dynamic" xmlns:exsl= "http://exslt.org/common" xmlns:func= "http://exslt.org/functions" xmlns:math= "http://exslt.org/math" xmlns:random= "http://exslt.org/random" xmlns:regexp= "http://exslt.org/regular-expressions" xmlns:set= "http://exslt.org/sets" xmlns:str= "http://exslt.org/strings" xmlns:dp= "http://www.datapower.com/extensions" extension-element-prefixes= "dp dyn" > <xsl:output method= "xml" omit-xml-declaration= "yes"/> <xsl:param name= "xpathpar" /> <xsl:template match= "/"> <xsl:variable name= "xpath"> <xsl:choose> <xsl:when test= "function-available('dp:http-request-header')"> <xsl:variable name= "auth" select= "dp:http-request-header('Authorization')"/> <xsl:variable name= "b64" select= "substring-after($auth,'Basic ')"/> <xsl:variable name= "user_" select= "dp:binary-decode($b64)"/> <xsl:value-of select= "substring($user_,1,string-length($user_)-1)"/> </xsl:when> <xsl:otherwise> <xsl:value-of select= "$xpathpar"/> </xsl:otherwise> </xsl:choose> </xsl:variable> <xsl:copy-of select= "dyn:evaluate($xpath)"/> </xsl:template> </xsl:stylesheet> <!--delim--> $ </pre>
    As answer on posting "Re: xsl Q: XSLT 1.0 output of namespace" on XSL-List I posted working template doOutput
    (publically available tomorrow) which can output namespace nodes as well as attribute nodes in a readable way.

    Find the new version of xpath++ tool making use of doOutput here:
    http://stamm-wilbrandt.de/en/xsl-list/xpath++

    This statement from the old version of xpath++
    
    <xsl:copy-of select=
    "dyn:evaluate($xpath)"/>
    


    got changed to this iteration over the results:
    
    <xsl:for-each select=
    "dyn:evaluate($xpath)"> <xsl:call-template name=
    "doOutput"/> </xsl:for-each>
    

    See here the
    • different (better) output for the attributes of previous posting's chess board example
    • a sample for output of namespaces
    • and a mixed example
    
    $ xpath++ 
    "/board/piece/@name | /board/piece/@color" position.xml   ------------------------------------------------------------------------------- name=
    "king" ------------------------------------------------------------------------------- color=
    "black" ------------------------------------------------------------------------------- name=
    "knight" ------------------------------------------------------------------------------- color=
    "white" ------------------------------------------------------------------------------- name=
    "king" ------------------------------------------------------------------------------- color=
    "white" ------------------------------------------------------------------------------- name=
    "bishop" ------------------------------------------------------------------------------- color=
    "white" $ $ xpath++ 
    "/a/c/*/namespace::*" d.xml   ------------------------------------------------------------------------------- xmlns:xml=
    "http://www.w3.org/XML/1998/namespace" ------------------------------------------------------------------------------- xmlns=
    "urn:d" $ $ xpath++ 
    "//attribute::* | /a/c/*/namespace::*" d.xml   ------------------------------------------------------------------------------- n1=
    "urn:a1" ------------------------------------------------------------------------------- a2=
    "a2" ------------------------------------------------------------------------------- xmlns:xml=
    "http://www.w3.org/XML/1998/namespace" ------------------------------------------------------------------------------- xmlns=
    "urn:d" $ $ cat d.xml <?some-PI testing?> <a> <!-- comment1 --> <b n1=
    "urn:a1" xmlns:n1=
    "urn:n1"/> <c>text1<d a2=
    "a2" xmlns=
    "urn:d"/></c> </a>   $
    


    Hermann.
  • HermannSW
    HermannSW
    4901 Posts

    Re: xpath tool

    ‏2010-08-18T08:59:18Z  
    • HermannSW
    • ‏2010-08-17T12:58:05Z
    As answer on posting "Re: xsl Q: XSLT 1.0 output of namespace" on XSL-List I posted working template doOutput
    (publically available tomorrow) which can output namespace nodes as well as attribute nodes in a readable way.

    Find the new version of xpath++ tool making use of doOutput here:
    http://stamm-wilbrandt.de/en/xsl-list/xpath++

    This statement from the old version of xpath++
    <pre class="jive-pre"> <xsl:copy-of select= "dyn:evaluate($xpath)"/> </pre>

    got changed to this iteration over the results:
    <pre class="jive-pre"> <xsl:for-each select= "dyn:evaluate($xpath)"> <xsl:call-template name= "doOutput"/> </xsl:for-each> </pre>
    See here the
    • different (better) output for the attributes of previous posting's chess board example
    • a sample for output of namespaces
    • and a mixed example
    <pre class="jive-pre"> $ xpath++ "/board/piece/@name | /board/piece/@color" position.xml ------------------------------------------------------------------------------- name= "king" ------------------------------------------------------------------------------- color= "black" ------------------------------------------------------------------------------- name= "knight" ------------------------------------------------------------------------------- color= "white" ------------------------------------------------------------------------------- name= "king" ------------------------------------------------------------------------------- color= "white" ------------------------------------------------------------------------------- name= "bishop" ------------------------------------------------------------------------------- color= "white" $ $ xpath++ "/a/c/*/namespace::*" d.xml ------------------------------------------------------------------------------- xmlns:xml= "http://www.w3.org/XML/1998/namespace" ------------------------------------------------------------------------------- xmlns= "urn:d" $ $ xpath++ "//attribute::* | /a/c/*/namespace::*" d.xml ------------------------------------------------------------------------------- n1= "urn:a1" ------------------------------------------------------------------------------- a2= "a2" ------------------------------------------------------------------------------- xmlns:xml= "http://www.w3.org/XML/1998/namespace" ------------------------------------------------------------------------------- xmlns= "urn:d" $ $ cat d.xml <?some-PI testing?> <a> <!-- comment1 --> <b n1= "urn:a1" xmlns:n1= "urn:n1"/> <c>text1<d a2= "a2" xmlns= "urn:d"/></c> </a> $ </pre>

    Hermann.
    Hello,

    yesterday's change was too quick.
    It was correct for results of dyn:evaluate() being node-set's only, so that for example "count(//*)" did not work anymore.
    The modified version is available here:
    http://stamm-wilbrandt.de/en/xsl-list/xpath++

    Now this
    <xsl:for-each select="dyn:evaluate($xpath)">
      <xsl:call-template name="doOutput"/>
    </xsl:for-each>
    


    has been modified to check for the result type:
    <xsl:variable name="result" select="dyn:evaluate($xpath)"/>
     
    <xsl:choose>
      <xsl:when test="exsl:object-type($result)='node-set'">
        <xsl:for-each select="$result">
          <xsl:call-template name="doOutput"/>
        </xsl:for-each>
      </xsl:when>
      <xsl:otherwise>
        <xsl:value-of select="$result"/>
      </xsl:otherwise>
    </xsl:choose>
    


    Additionally added "xmlns:xs" namespace declaration.
    Updated on 2014-03-25T03:40:58Z at 2014-03-25T03:40:58Z by iron-man
  • Jaango
    Jaango
    267 Posts

    Re: xpath tool

    ‏2010-12-09T10:44:39Z  
    • HermannSW
    • ‏2010-08-18T08:59:18Z
    Hello,

    yesterday's change was too quick.
    It was correct for results of dyn:evaluate() being node-set's only, so that for example "count(//*)" did not work anymore.
    The modified version is available here:
    http://stamm-wilbrandt.de/en/xsl-list/xpath++

    Now this
    <pre class="java dw" data-editor-lang="java" data-pbcklang="java" dir="ltr"><xsl:for-each select="dyn:evaluate($xpath)"> <xsl:call-template name="doOutput"/> </xsl:for-each> </pre>

    has been modified to check for the result type:
    <pre class="java dw" data-editor-lang="java" data-pbcklang="java" dir="ltr"><xsl:variable name="result" select="dyn:evaluate($xpath)"/> <xsl:choose> <xsl:when test="exsl:object-type($result)='node-set'"> <xsl:for-each select="$result"> <xsl:call-template name="doOutput"/> </xsl:for-each> </xsl:when> <xsl:otherwise> <xsl:value-of select="$result"/> </xsl:otherwise> </xsl:choose> </pre>

    Additionally added "xmlns:xs" namespace declaration.
    Hi All,

    Please help me with the below problem.

    1. ./xpath++.sh "count(//*)" x.xml
    ./xpath++.sh: line 18: xsltproc: command not found

    I was executing this command in a linux server. However getting the above error. How I have done is
    Saved the lates file as with.sh extension that is xpath++.sh
    Saved the file x.xml, present in this thread at the start
    Executed the shell script as above.
    Any help on this.
  • HermannSW
    HermannSW
    4901 Posts

    Re: xpath tool

    ‏2010-12-11T08:46:17Z  
    • Jaango
    • ‏2010-12-09T10:44:39Z
    Hi All,

    Please help me with the below problem.

    1. ./xpath++.sh "count(//*)" x.xml
    ./xpath++.sh: line 18: xsltproc: command not found

    I was executing this command in a linux server. However getting the above error. How I have done is
    Saved the lates file as with.sh extension that is xpath++.sh
    Saved the file x.xml, present in this thread at the start
    Executed the shell script as above.
    Any help on this.
    Hi,

    > ...
    > # ./xpath++.sh "count(//*)" x.xml
    > ./xpath++.sh: line 18: xsltproc: command not found
    >
    you see the error because xsltproc XSLT processor is not installed on your system.

    Either install xsltproc (part of libexslt).

    Or just use your DataPower box as XSLT processor.

    You have to make (one time) two changes in the script and install the stylesheet on a DataPower box.

    1. comment out the "xsltproc_xsl...." line:
    
    # xsltproc_xsl=/tmp/temp.xsl
    


    2. change the URL to match your DataPower box and service:
    
    url=http:
    //dp1-l3.boeblingen.de.ibm.com:2048/xpath
    


    3. Extract the stylesheet between the two "<delim>" elements of the bottom of the script
    and install it on the DataPower box refered to by the url from 2.

    
    <!--delim--> <xsl:stylesheet version=
    "1.0" xmlns:xsl=
    "http://www.w3.org/1999/XSL/Transform" xmlns:date=
    "http://exslt.org/dates-and-times" ... <xsl:copy-of select=
    "dyn:evaluate($xpath)"/> </xsl:template> </xsl:stylesheet> <!--delim-->
    


    Btw, this stylesheet is an example for cross XSLT-Processor XSLT
    (DataPower does not provide "--stringparam" of xsltproc, and xsltproc
    does not allow to access HTTP headers.

    Hermann.
  • Jaango
    Jaango
    267 Posts

    Re: xpath tool

    ‏2010-12-13T15:29:01Z  
    • HermannSW
    • ‏2010-12-11T08:46:17Z
    Hi,

    > ...
    > # ./xpath++.sh "count(//*)" x.xml
    > ./xpath++.sh: line 18: xsltproc: command not found
    >
    you see the error because xsltproc XSLT processor is not installed on your system.

    Either install xsltproc (part of libexslt).

    Or just use your DataPower box as XSLT processor.

    You have to make (one time) two changes in the script and install the stylesheet on a DataPower box.

    1. comment out the "xsltproc_xsl...." line:
    <pre class="jive-pre"> # xsltproc_xsl=/tmp/temp.xsl </pre>

    2. change the URL to match your DataPower box and service:
    <pre class="jive-pre"> url=http: //dp1-l3.boeblingen.de.ibm.com:2048/xpath </pre>

    3. Extract the stylesheet between the two "<delim>" elements of the bottom of the script
    and install it on the DataPower box refered to by the url from 2.

    <pre class="jive-pre"> <!--delim--> <xsl:stylesheet version= "1.0" xmlns:xsl= "http://www.w3.org/1999/XSL/Transform" xmlns:date= "http://exslt.org/dates-and-times" ... <xsl:copy-of select= "dyn:evaluate($xpath)"/> </xsl:template> </xsl:stylesheet> <!--delim--> </pre>

    Btw, this stylesheet is an example for cross XSLT-Processor XSLT
    (DataPower does not provide "--stringparam" of xsltproc, and xsltproc
    does not allow to access HTTP headers.

    Hermann.
    Hi Hermann,

    I get now, curl commad not found error.

    I referenced the xpath xsl file by creating a simple XMLFW service in loopback mode and commented out the xsltproc line in the script. Is there aything missing in my linux server.
  • HermannSW
    HermannSW
    4901 Posts

    Re: xpath tool

    ‏2010-12-13T23:58:18Z  
    • Jaango
    • ‏2010-12-13T15:29:01Z
    Hi Hermann,

    I get now, curl commad not found error.

    I referenced the xpath xsl file by creating a simple XMLFW service in loopback mode and commented out the xsltproc line in the script. Is there aything missing in my linux server.
    > I get now, curl commad not found error.
    >
    > I referenced the xpath xsl file by creating a simple XMLFW service in loopback mode and commented out the xsltproc line in the script. Is there aything missing in my linux server.

    curl is used in this part of the script:
    ...
    else
      url=http://dp1-l3.boeblingen.de.ibm.com:2048/xpath
     
      curl -u "$1:" --data-binary @$2 $url; echo
    fi
    ...
    

    It sends the request to the DataPower box (XPath expression=$1, XML File=$2).
    You may get it from http://curl.haxx.se/, it is free and open software.

    Hermann.
    Updated on 2014-03-25T03:36:42Z at 2014-03-25T03:36:42Z by iron-man
  • Jaango
    Jaango
    267 Posts

    Re: xpath tool

    ‏2010-12-15T09:54:50Z  
    • HermannSW
    • ‏2010-12-13T23:58:18Z
    > I get now, curl commad not found error.
    >
    > I referenced the xpath xsl file by creating a simple XMLFW service in loopback mode and commented out the xsltproc line in the script. Is there aything missing in my linux server.

    curl is used in this part of the script:
    <pre class="java dw" data-editor-lang="java" data-pbcklang="java" dir="ltr">... else url=http://dp1-l3.boeblingen.de.ibm.com:2048/xpath curl -u "$1:" --data-binary @$2 $url; echo fi ... </pre>
    It sends the request to the DataPower box (XPath expression=$1, XML File=$2).
    You may get it from http://curl.haxx.se/, it is free and open software.

    Hermann.
    Perfect. Thanks a lot
  • Jaango
    Jaango
    267 Posts

    Re: xpath tool

    ‏2011-03-17T13:23:27Z  
    • Jaango
    • ‏2010-12-15T09:54:50Z
    Perfect. Thanks a lot
    Hi Hermann,

    Today I tried the above in Windows machine, Cygwin. It is working in a unix server. However I got the below error. What should be the extension to be used to save the xpath++ file. I saved it with .sh extension. What am i missing here.
    $ ./xpath++.sh "count(//*)" x.xml
    ./xpath++.sh: line 28: syntax error: unexpected end of file
  • HermannSW
    HermannSW
    4901 Posts

    Re: xpath tool

    ‏2011-03-17T15:31:02Z  
    • Jaango
    • ‏2011-03-17T13:23:27Z
    Hi Hermann,

    Today I tried the above in Windows machine, Cygwin. It is working in a unix server. However I got the below error. What should be the extension to be used to save the xpath++ file. I saved it with .sh extension. What am i missing here.
    $ ./xpath++.sh "count(//*)" x.xml
    ./xpath++.sh: line 28: syntax error: unexpected end of file
    Hi,

    not sure what is wrong, for me it works with and without ".sh" suffix.
    Please do the following steps (the first two check preconditions).
    If xsltproc is missing, please install libxslt via (cygwin) setup.exe
    
    $ [cygwin] which sed /usr/bin/sed $ [cygwin] which xsltproc /usr/bin/xsltproc $ [cygwin] $ [cygwin] rm -f xpath++* $ [cygwin] wget -q http:
    //stamm-wilbrandt.de/en/xsl-list/xpath++ $ [cygwin] chmod 755 xpath++ $ [cygwin] $ [cygwin] echo 
    "<a><b/><c/></a>" | ./xpath++ 
    "count(//*)" - 3 $ [cygwin]
    


    The execution shows a nice feature, you may use the input redirect (-) as XML "file".

    Echoing a "dummy" XML file allows to execute the exslt.org samples without change!
    (because the exslt prefixes are defined in xpath++ )

    From http://www.exslt.org/str/functions/split/index.html:
    
    $ [cygwin] echo 
    "<dummy/>" | ./xpath++ 
    "str:split('a, simple, list', ', ')" -   ------------------------------------------------------------------------------- <token>a</token> ------------------------------------------------------------------------------- <token>simple</token> ------------------------------------------------------------------------------- <token>list</token> $ [cygwin]
    


    Or you may apply xpath++ to result received by XML Management request:
    
    $ curl -k -s -u admin --data-binary @version.xml https:
    //dp5-l3:5550/service/mgmt/current | xpath++ "//*[local-name() = 'RunningDPOS']/text()" - Enter host password 
    
    for user 
    'admin': XI50.3.8.2.2 $ curl -k -s -u admin --data-binary @version.xml https:
    //dp5-l3:5550/service/mgmt/current | xpath++ "//*[local-name() = 'XMLAccelerator']/text()" - Enter host password 
    
    for user 
    'admin': xg4ng $ $ cat version.xml <?xml version=
    "1.0" encoding=
    "UTF-8"?> <env:Envelope xmlns:env=
    "http://schemas.xmlsoap.org/soap/envelope/"> <env:Body> <dp:request xmlns:dp=
    "http://www.datapower.com/schemas/management"> <dp:get-status class=
    "FirmwareVersion"/> </dp:request> </env:Body> </env:Envelope> $
    


    Or determine the list of SOMA commands ...
    https://www.ibm.com/developerworks/forums/thread.jspa?messageID=14594278#14594291

     
    Hermann<myXsltBlog/>
  • Jaango
    Jaango
    267 Posts

    Re: xpath tool

    ‏2011-03-18T07:32:11Z  
    • HermannSW
    • ‏2011-03-17T15:31:02Z
    Hi,

    not sure what is wrong, for me it works with and without ".sh" suffix.
    Please do the following steps (the first two check preconditions).
    If xsltproc is missing, please install libxslt via (cygwin) setup.exe
    <pre class="jive-pre"> $ [cygwin] which sed /usr/bin/sed $ [cygwin] which xsltproc /usr/bin/xsltproc $ [cygwin] $ [cygwin] rm -f xpath++* $ [cygwin] wget -q http: //stamm-wilbrandt.de/en/xsl-list/xpath++ $ [cygwin] chmod 755 xpath++ $ [cygwin] $ [cygwin] echo "<a><b/><c/></a>" | ./xpath++ "count(//*)" - 3 $ [cygwin] </pre>

    The execution shows a nice feature, you may use the input redirect (-) as XML "file".

    Echoing a "dummy" XML file allows to execute the exslt.org samples without change!
    (because the exslt prefixes are defined in xpath++ )

    From http://www.exslt.org/str/functions/split/index.html:
    <pre class="jive-pre"> $ [cygwin] echo "<dummy/>" | ./xpath++ "str:split('a, simple, list', ', ')" - ------------------------------------------------------------------------------- <token>a</token> ------------------------------------------------------------------------------- <token>simple</token> ------------------------------------------------------------------------------- <token>list</token> $ [cygwin] </pre>

    Or you may apply xpath++ to result received by XML Management request:
    <pre class="jive-pre"> $ curl -k -s -u admin --data-binary @version.xml https: //dp5-l3:5550/service/mgmt/current | xpath++ "//*[local-name() = 'RunningDPOS']/text()" - Enter host password for user 'admin': XI50.3.8.2.2 $ curl -k -s -u admin --data-binary @version.xml https: //dp5-l3:5550/service/mgmt/current | xpath++ "//*[local-name() = 'XMLAccelerator']/text()" - Enter host password for user 'admin': xg4ng $ $ cat version.xml <?xml version= "1.0" encoding= "UTF-8"?> <env:Envelope xmlns:env= "http://schemas.xmlsoap.org/soap/envelope/"> <env:Body> <dp:request xmlns:dp= "http://www.datapower.com/schemas/management"> <dp:get-status class= "FirmwareVersion"/> </dp:request> </env:Body> </env:Envelope> $ </pre>

    Or determine the list of SOMA commands ...
    https://www.ibm.com/developerworks/forums/thread.jspa?messageID=14594278#14594291

     
    Hermann<myXsltBlog/>
    Perfect.

    It is working now in windows after I ran dos2unix command.
  • HermannSW
    HermannSW
    4901 Posts

    Re: xpath tool

    ‏2011-10-18T19:22:13Z  
    Posting "Re: coproc2 / piggyback.xsl"
    https://www.ibm.com/developerworks/forums/thread.jspa?messageID=14694522#14694522

    provides a xpath++ solution which does not require installation of xpath++ endpoint on DataPower
    (stylesheet from this posting installed as service) if coproc2 service is already installed.

    While that solution is a nice demonstration of piggyback technique for coproc2,
    a "normal" stylesheet implementation is preferable for a xpath++ coproc2 service.
    Stylesheet xpath++.xsl is just a slight simplification of the dual use (DataPower and xsltproc) stylesheet
    of xpath++ tool (between the two <!--delim--> comments).

    By coproc2.java Java client xpath++ tool is now usable on all (Java) platforms!
    
    $ cat ab.xml <a>1<b>2</b>3</a> $ $ java coproc2 xpath++.xsl ab.xml 
    "http://dp2-l3:2223?//*"; echo   ------------------------------------------------------------------------------- <a>1<b>2</b>3</a> ------------------------------------------------------------------------------- <b>2</b>   $
    


    This is stylesheet xpath++.xsl (also attached):
    
    <xsl:stylesheet version=
    "1.0" xmlns:xsl=
    "http://www.w3.org/1999/XSL/Transform" xmlns:xs=
    "http://www.w3.org/2001/XMLSchema" xmlns:date=
    "http://exslt.org/dates-and-times" xmlns:dyn=
    "http://exslt.org/dynamic" xmlns:exsl=
    "http://exslt.org/common" xmlns:func=
    "http://exslt.org/functions" xmlns:math=
    "http://exslt.org/math" xmlns:random=
    "http://exslt.org/random" xmlns:regexp=
    "http://exslt.org/regular-expressions" xmlns:set=
    "http://exslt.org/sets" xmlns:str=
    "http://exslt.org/strings" xmlns:dp=
    "http://www.datapower.com/extensions" extension-element-prefixes=
    "dp dyn" > <xsl:output method=
    "xml" omit-xml-declaration=
    "yes"/> <xsl:template match=
    "/"> <xsl:variable name=
    "xpath" select=
    "substring-after(dp:variable('var://service/URI'),'?')"/> <xsl:variable name=
    "result" select=
    "dyn:evaluate($xpath)"/>   <xsl:choose> <xsl:when test=
    "exsl:object-type($result)='node-set'"> <xsl:for-each select=
    "$result"> <xsl:call-template name=
    "doOutput"/> </xsl:for-each> </xsl:when> <xsl:otherwise> <xsl:value-of select=
    "$result"/> </xsl:otherwise> </xsl:choose> </xsl:template>     <xsl:template name=
    "doOutput"> <xsl:text> ------------------------------------------------------------------------------- </xsl:text> <xsl:choose> <xsl:when test=
    "self::comment() |  self::processing-instruction() | self::text()
    "> <xsl:copy-of select=
    "."/> </xsl:when> <xsl:when test=
    "count(. | ../attribute::* | ../namespace::*) != count(../attribute::* | ../namespace::*)
    "> <xsl:copy-of select=
    "."/> </xsl:when> <xsl:otherwise> <xsl:
    
    if test=
    "count(. | ../namespace::*) = count(../namespace::*)"> <xsl:text>xmlns</xsl:text> <xsl:
    
    if test=
    "name()">:</xsl:if> </xsl:if> <xsl:value-of select=
    "name()"/> <xsl:text>=
    "</xsl:text> <xsl:value-of select=
    "string()"/> <xsl:text>
    "</xsl:text> </xsl:otherwise> </xsl:choose> </xsl:template> </xsl:stylesheet>
    


     
    Hermann<myXsltBlog/>
  • HermannSW
    HermannSW
    4901 Posts

    Re: xpath tool

    ‏2011-10-25T17:12:32Z  
    xpath++ is especially useful when working on the "correct" regexp statement for a given task.

    Below is a demonstration on how to match linefeed and tab characters ...
    $ echo "<x>&​#9;b&​#10;B</x>" | \
    > java coproc2 xpath++.xsl - http://dp2-l3:2223?\
    > "regexp:replace(.,'b','gi','Z')"; echo
            Z
    Z
     
    $ echo "<x>&​#9;b&​#10;B</x>" | \
    > java coproc2 xpath++.xsl - http://dp2-l3:2223?\
    > "regexp:replace(.,'\t','gi','Z')"; echo
    Zb
    B
     
    $ echo "<x>&​#9;b&​#10;B</x>" | \
    > java coproc2 xpath++.xsl - http://dp2-l3:2223?\
    > "regexp:replace(.,'\n','gi','Z')"; echo
            bZB
     
    $
    


     
    Hermann <myXsltBlog/>
    Updated on 2014-03-25T03:07:53Z at 2014-03-25T03:07:53Z by iron-man
  • Jaango
    Jaango
    267 Posts

    Re: xpath tool

    ‏2011-11-09T13:45:58Z  
    • HermannSW
    • ‏2011-10-25T17:12:32Z
    xpath++ is especially useful when working on the "correct" regexp statement for a given task.

    Below is a demonstration on how to match linefeed and tab characters ...
    <pre class="java dw" data-editor-lang="java" data-pbcklang="java" dir="ltr">$ echo "<x>&​#9;b&​#10;B</x>" | \ > java coproc2 xpath++.xsl - http://dp2-l3:2223?\ > "regexp:replace(.,'b','gi','Z')"; echo Z Z $ echo "<x>&​#9;b&​#10;B</x>" | \ > java coproc2 xpath++.xsl - http://dp2-l3:2223?\ > "regexp:replace(.,'\t','gi','Z')"; echo Zb B $ echo "<x>&​#9;b&​#10;B</x>" | \ > java coproc2 xpath++.xsl - http://dp2-l3:2223?\ > "regexp:replace(.,'\n','gi','Z')"; echo bZB $ </pre>

     
    Hermann <myXsltBlog/>
    Hi Hermann,

    When I ran the above, got the below error. Do I have to change the request type for the coproc2 service?

    illegal character 0xe2 at offset 4 of http://boxip:2223?
  • HermannSW
    HermannSW
    4901 Posts

    Re: xpath tool

    ‏2011-11-10T14:39:35Z  
    • Jaango
    • ‏2011-11-09T13:45:58Z
    Hi Hermann,

    When I ran the above, got the below error. Do I have to change the request type for the coproc2 service?

    illegal character 0xe2 at offset 4 of http://boxip:2223?
    Hi,
    >
    > When I ran the above, got the below error. Do I have to change the request type for the coproc2 service?
    >
    > illegal character 0xe2 at offset 4 of http://boxip:2223?
    >
    ah, you got me!

    In order to get the display right and make forum software not replace &​#9; by the tab character
    I inserted the zero width space character &​#8203; between the ampersand and the rest of the entity.
    The UTF-8 encoding of unicode character 8203 is E2808B.
    
    $ echo 
    "...x>&​#9;..." | od -Ax -tcx1 000000   .   .   .   x   >   & 342 200 213   #   9   ;   .   .   .  \n 2e  2e  2e  78  3e  26  e2  80  8b  23  39  3b  2e  2e  2e  0a 000010 $
    


    The "0xe2" is exactly what your error message complained about.

    Find attached the commands without zero width space character for copy and paste.
    Below you can see that now your browser interprets the tab and linefeed characters, even in {​code}...{​code} mode of this Forum:
    
    $ echo 
    "<x>	b
    B</x>" | \ > java coproc2 xpath++.xsl - http:
    //dp2-l3:2223?\ > 
    "regexp:replace(.,'b','gi','Z')"; echo Z Z   $
    


    Hermann<myXsltBlog/>
  • Jaango
    Jaango
    267 Posts

    Re: xpath tool

    ‏2011-11-11T11:24:37Z  
    • HermannSW
    • ‏2011-10-18T19:22:13Z
    Posting "Re: coproc2 / piggyback.xsl"
    https://www.ibm.com/developerworks/forums/thread.jspa?messageID=14694522#14694522

    provides a xpath++ solution which does not require installation of xpath++ endpoint on DataPower
    (stylesheet from this posting installed as service) if coproc2 service is already installed.

    While that solution is a nice demonstration of piggyback technique for coproc2,
    a "normal" stylesheet implementation is preferable for a xpath++ coproc2 service.
    Stylesheet xpath++.xsl is just a slight simplification of the dual use (DataPower and xsltproc) stylesheet
    of xpath++ tool (between the two <!--delim--> comments).

    By coproc2.java Java client xpath++ tool is now usable on all (Java) platforms!
    <pre class="jive-pre"> $ cat ab.xml <a>1<b>2</b>3</a> $ $ java coproc2 xpath++.xsl ab.xml "http://dp2-l3:2223?//*"; echo ------------------------------------------------------------------------------- <a>1<b>2</b>3</a> ------------------------------------------------------------------------------- <b>2</b> $ </pre>

    This is stylesheet xpath++.xsl (also attached):
    <pre class="jive-pre"> <xsl:stylesheet version= "1.0" xmlns:xsl= "http://www.w3.org/1999/XSL/Transform" xmlns:xs= "http://www.w3.org/2001/XMLSchema" xmlns:date= "http://exslt.org/dates-and-times" xmlns:dyn= "http://exslt.org/dynamic" xmlns:exsl= "http://exslt.org/common" xmlns:func= "http://exslt.org/functions" xmlns:math= "http://exslt.org/math" xmlns:random= "http://exslt.org/random" xmlns:regexp= "http://exslt.org/regular-expressions" xmlns:set= "http://exslt.org/sets" xmlns:str= "http://exslt.org/strings" xmlns:dp= "http://www.datapower.com/extensions" extension-element-prefixes= "dp dyn" > <xsl:output method= "xml" omit-xml-declaration= "yes"/> <xsl:template match= "/"> <xsl:variable name= "xpath" select= "substring-after(dp:variable('var://service/URI'),'?')"/> <xsl:variable name= "result" select= "dyn:evaluate($xpath)"/> <xsl:choose> <xsl:when test= "exsl:object-type($result)='node-set'"> <xsl:for-each select= "$result"> <xsl:call-template name= "doOutput"/> </xsl:for-each> </xsl:when> <xsl:otherwise> <xsl:value-of select= "$result"/> </xsl:otherwise> </xsl:choose> </xsl:template> <xsl:template name= "doOutput"> <xsl:text> ------------------------------------------------------------------------------- </xsl:text> <xsl:choose> <xsl:when test= "self::comment() | self::processing-instruction() | self::text() "> <xsl:copy-of select= "."/> </xsl:when> <xsl:when test= "count(. | ../attribute::* | ../namespace::*) != count(../attribute::* | ../namespace::*) "> <xsl:copy-of select= "."/> </xsl:when> <xsl:otherwise> <xsl: if test= "count(. | ../namespace::*) = count(../namespace::*)"> <xsl:text>xmlns</xsl:text> <xsl: if test= "name()">:</xsl:if> </xsl:if> <xsl:value-of select= "name()"/> <xsl:text>= "</xsl:text> <xsl:value-of select= "string()"/> <xsl:text> "</xsl:text> </xsl:otherwise> </xsl:choose> </xsl:template> </xsl:stylesheet> </pre>

     
    Hermann<myXsltBlog/>
    Hermann,
    Thanks a lot. This is a great way to test regex expressions. I think i need to read about utf8 coversions. :)