This article offers five tips—derived from real-world applications for which hours were spent digging deep into confusing and unexpected behaviors of XPath—for using XPath in your work:

• False is sometimes true.
• The XPath expressions (x != y) and not(x = y) are different and give different results.
• The value of the XPath position() function changes depending on its context.
• Select the first element of a given name with XPath.
• Debug XPath select expressions that fail because of default namespaces.

With these tips, you can spare yourself hours of frustration.

Tip 1: False is sometimes true

A string whose length is greater than zero is always evaluated as True when resolved to a Boolean function, even if its value is the string 'false'.

In an XPath expression, comparing a string and a Boolean function can yield unexpected results. To avoid trouble, do not compare a string to the built-in Boolean functions true() and false(). Instead, compare strings to the string values 'true' and 'false'. Keeping the Boolean functions true() and false() as Booleans without having them unintentionally converted to strings is difficult. A non-empty string is considered True even if its value is 'false'; therefore, boolean('false') = true() and boolean('true') = true(). Only an empty string ('') is False—namely, boolean('') = false().

Consider the unexpected result when a Boolean function is converted to a string and back again. First, the Boolean is converted to a string (for example, string(false()) = 'false'). When converted back, as in boolean(string(false())), the result is true().

The string() function, however, is not the only means of converting a Boolean function to a string. Result tree fragments (RTF)—which are sometimes unavoidable—are strings. For example, consider Listing 1, which is an XML document of documents with their name, year of publication, and number of pages.

<Documents>
  <Document id="18396">
    <Name>Knowing the Good</Name>
    <Year>2010</Year>
    <Pages>35</Pages>
  </Document>
  <Document id="18397">
    <Name>Beyond Standards: Best Practices</Name>
    <Year>2011</Year>
    <Pages>12</Pages>
  </Document>
  <Document id="18398">
    <Name>101 Ways to Do the Same Thing</Name>
    <Year>2011</Year>
    <Pages>50</Pages>
  </Document>
</Documents>

Listing 2 is an XSLT template that searches for recent documents with at least 20 pages. Unfortunately, this template produces incorrect results.

<xsl:template match="Document">
  <xsl:variable name="recentFullLengthPublications">
    <xsl:choose>
      <xsl:when test="Year=2011 and Pages &gt;= 20">
        <xsl:value-of select="true()"/>
      </xsl:when>
      <xsl:otherwise>
        <xsl:value-of select="false()"/>
      </xsl:otherwise>
    </xsl:choose>
  </xsl:variable>
  <xsl:if test="$recentFullLengthPublications">
    <xsl:copy-of select="Name"/>
  </xsl:if>
</xsl:template>

The transformation returns the XML in Listing 3, which is incorrect. Only one element should be returned—namely, "101 Ways to Do the Same Thing." All three elements, however, are returned.

<Name>Knowing the Good</Name>
<Name>Beyond Standards: Best Practices</Name>
<Name>101 Ways to Do the Same Thing</Name>

The document named "101 Ways to Do the Same Thing" is the only document in 2011 with at least 20 pages, so what went wrong? Content of a variable produces an RTF. This fragment is evaluated as a string, even if its value was a Boolean function.

The test expression in <xsl:if test="$recentFullLengthPublications"> from Listing 2 is incorrect. The test is always True; all <Name> elements in the source document are returned. Even the test expression in <xsl:if test="$recentFullLengthPublications=true()"> is incorrect. The variable $recentFullLengthPublications is going to be either the string 'true' or 'false', not a Boolean data type. Therefore, the statement <xsl:if test="$recentFullLengthPublications='true'"> does work.

The template, however, is confusing because it uses the true() and false() functions, which are misleading in this case. The template in Listing 4 is both correct and more readable. It uses 'yes' and 'no' instead of 'true' and 'false' to avoid the association with Boolean types. Remember to enclose 'yes' and 'no' in single quotation marks.

<xsl:template match="Document">
  <xsl:variable name="recentFullLengthPublications">
    <xsl:choose>
      <xsl:when test="Year=2011 and Pages &gt;= 20">
        <xsl:value-of select="'yes'"/>
      </xsl:when>
      <xsl:otherwise>
        <xsl:value-of select="'no'"/>
      </xsl:otherwise>
    </xsl:choose>
  </xsl:variable>
  <xsl:if test="$recentFullLengthPublications='yes'">
    <xsl:copy-of select="Name"/>
  </xsl:if>
</xsl:template>

Beware the Boolean functions true() and false(). In this simple example, the variable is not necessary and, even if used, could have been defined with a select attribute. The result with a select attribute is not automatically converted to a string; as such, a Boolean value remains a Boolean. The select attribute does not return an RTF.

Often, a condition is too complex for the select attribute, and you must define a variable as an RTF. For this simple example, either of the templates in Listing 5 and Listing 6 are preferred. In Listing 5, the test does not need to compare to a string because the variable returns a Boolean type rather than an RTF as a result of the select attribute.

<xsl:template match="Document">
  <xsl:variable name="recentFullLengthPublications" 
        select="Year=2011 and Pages &gt;= 20" />
  <xsl:if test="$recentFullLengthPublications">
    <xsl:copy-of select="Name"/>
  </xsl:if>
</xsl:template>

Alternatively, you can drop the variable, as in Listing 6, because the condition is so simple. The variable, however, is self-documenting.

<xsl:template match="Document">
  <xsl:if test="Year=2011 and Pages &gt;= 20">
    <xsl:copy-of select="Name"/>
  </xsl:if>
</xsl:template>

Back to top

Tip 2: Know the difference between (x != y) and not(x = y)

The XPath expressions (x != y) and not(x = y) are different.

As a quick summary:

• (x != y) selects nodes with both x and y, and their values are different. This expression is more exclusive than not(x = y) because the node must have x and y.
• not(x = y) selects nodes missing either x or y or that have x and y, but their values are different. This expression is more inclusive.

With most computer languages, the expressions x != y and not(x = y) are equivalent; with XML, though, values can be undefined because a node is missing. When XML node values are compared, a question arises: How should missing values be evaluated? For a simple equality comparison, consider x = y: If either x or y is missing, the result is false. Another question is, how should the expression be negated? The following example answers this question.

Listing 7 is an XML document of documents. The data can be queried to find documents published in a given year or within a range of pages. The last document—"Prospective Projects"—does not have a year or number of pages. This document is incomplete and therefore has not been published.

<Documents>
  <Document id="18396">
    <Name>Knowing the Good</Name>
    <Year>2010</Year>
    <Pages>35</Pages>
  </Document>
  <Document id="18397">
    <Name>Beyond Standards: Best Practices</Name>
    <Year>2011</Year>
    <Pages>12</Pages>
  </Document>
  <Document id="18398">
    <Name>101 Ways to Do the Same Thing</Name>
    <Year>2011</Year>
    <Pages>50</Pages>
  </Document>
  <Document id="18399">
    <Name>Prospective Projects</Name>
  </Document>
</Documents>

The XPath expression //Document[Year = 2011]/Name selects documents published in 2011. Listing 8 shows the result is the two documents.

<Name>Beyond Standards: Best Practices</Name>
<Name>101 Ways to Do the Same Thing</Name>

Next, consider how to select documents that were not published in 2011. Two choices exist to negate an XPath expression: not() and !=. All documents not in the set of 2011 documents are selected with the XPath expression //Document[not(Year = 2011)]/Name. Listing 9 shows the result is the other two documents.

<Name>Knowing the Good</Name>
<Name>Prospective Projects</Name>

To select all documents that have been published but not in 2011, use the expression //Document[Year != 2011]/Name. The result is Listing 10, which shows just a single document.

<Name>Knowing the Good</Name>

As shown, the two XPath expressions not(Year = 2011) and Year != 2011 have slightly different meanings. You use the XPath expression //Document[Pages >= 20]/Name to get all the documents with 20 or more pages. The result is Listing 11, which shows both documents matching the condition.

<Name>Knowing the Good</Name>
<Name>101 Ways to Do the Same Thing</Name>

Next, consider the opposite: documents with less than 20 pages. The expression //Document[Pages < 20]/Name selects these documents. The sole matching document is the result in Listing 12.

<Name>Beyond Standards: Best Practices</Name>

Finally, note that "Prospective Projects" does not have a known number of pages. Include such documents with not() instead of !=—for example, //Document[not(Pages >= 20)]/Name. The result is Listing 13, with two documents.

<Name>Beyond Standards: Best Practices</Name>
<Name>Prospective Projects</Name>

To get the list of documents without a known number of pages, use the XPath expression //Document[not(Pages)]/Name. Listing 14 shows the document matching the condition.

<Name>Prospective Projects</Name>

The not() function is essential for handling nodes with missing attributes or elements. See Resources for a link to more tips on using XPath.

Back to top

Tip 3: The value of position() depends on where it is used

The value of the position() function depends on its context—where it is used. This dependence is especially true when it's used in a predicate.

The position() function returns the one-based index of the node within the node set. The position() function counts nodes—both elements and text nodes—even if the text is just white space. The following examples illustrate position() in several different contexts.

Listing 15 is an XML document of documents and their authors. Of interest are the <Name> elements.

<Documents>
  <Document id="18396">
    <Name>Knowing the Good</Name>
    <Authors>
      <Name>Bob Bloggs</Name>
      <Name>Kim Bergess</Name>
    </Authors>
  </Document>
  <Document id="18397">
    <Name>Beyond Standards: Best Practices</Name>
    <Authors>
      <Name>Bill Agnew</Name>
    </Authors>
  </Document>
  <Document id="18398">
    <Name>101 Ways to Do the Same Thing</Name>
    <Authors>
      <Name>Jay Nerks</Name>
      <Name>Paula Towne</Name>
      <Name>Carol Tinney</Name>
    </Authors>
  </Document>
</Documents>

The XSLT template in Listing 16 displays the position of each <Name> element. Listing 17 shows the results.

<xsl:template match="Name">
  <xsl:value-of select="concat(position(),' ',.,'&#10;')"/>
</xsl:template>

The first column is the value of the position() function. Listing 17 shows the results of the XSLT template transform.

2 Knowing the Good
2 Bob Bloggs
4 Kim Bergess
2 Beyond Standards: Best Practices
2 Bill Agnew
2 101 Ways to Do the Same Thing
2 Jay Nerks
4 Paula Towne
6 Carol Tinney

The results list the position of the <Name> element within the context of the template. Even though most name elements are the first element within their parent element, they are the second node because a blank text node (including a line break) exists between the tags. In addition, the position is relative to its parent element, not the whole document.

The templates in Listing 18 isolate the <Name> elements from the text nodes.

<xsl:template match="Documents">
  <xsl:apply-templates select="//Name"/>
</xsl:template>

<xsl:template match="Name">
  <xsl:value-of select="concat(position(),' ',.,'&#10;')"/>
</xsl:template>

The result with white-space text nodes ignored is Listing 19. The value of the position() function is shown in the first column.

1 Knowing the Good
2 Bob Bloggs
3 Kim Bergess
4 Beyond Standards: Best Practices
5 Bill Agnew
6 101 Ways to Do the Same Thing
7 Jay Nerks
8 Paula Towne
9 Carol Tinney

The first template selects and processes all of the <Name> elements. Because all nine of the <Name> elements are selected as a set, the position() is the index of an element within that set.

In the next example, the position() function is used in the predicate for the xsl:apply-templates statement, which selects names that have a position of 1. See Listing 20.

<xsl:template match="Documents">
  <xsl:apply-templates select="//Name[position()=1]"/>
</xsl:template>

<xsl:template match="Name">
  <xsl:value-of select="concat(position(),' ',.,'&#10;')"/>
</xsl:template>

The result is Listing 21. Notice that only six items are returned. The first column is the value of the position() function of those Name elements that are first within their parent element.

1 Knowing the Good
2 Bob Bloggs
3 Beyond Standards: Best Practices
4 Bill Agnew
5 101 Ways to Do the Same Thing
6 Jay Nerks

Note that only the first author is included. When you use position() in the predicate—that is, within the square brackets ([])—the index is relative to the <Name> element's parent. Only the first <Name> element within the <Authors> element is returned. As a shortcut, Name[position()=1] can be expressed as Name[1].

Listing 22 includes a template with the position() function placed in the template's match expression rather than in the xsl:apply-templates statement. The third template discards Name elements not in position 1.

<xsl:template match="Documents">
  <xsl:apply-templates select="//Name"/>
</xsl:template>

<xsl:template match="Name[position()=1]">
  <xsl:value-of select="concat(position(),' ',.,'&#10;')"/>
</xsl:template>

<xsl:template match="Name" />

The results provided in Listing 23 are the same six elements as before, but the positions are different.

1 Knowing the Good
2 Bob Bloggs
4 Beyond Standards: Best Practices
5 Bill Agnew
6 101 Ways to Do the Same Thing
7 Jay Nerks

The names in Listing 23 are the same as in Listing 21, but the position values are different. All nine names are applied to the template, but only six are matched, whereas in Listing 20, only six are applied to the template. As a shortcut, Name[position()=1] can be expressed as Name[1].

Listing 24 shows a template to process only the first <Name> element in the XML document.

<xsl:template match="Documents">
  <xsl:apply-templates select="(//Name)[1]"/>
</xsl:template>

<xsl:template match="Name">
  <xsl:value-of select="concat(position(),' ',.,'&#10;')"/>
</xsl:template>

Listing 25 shows the single result.

1 Knowing the Good

The parentheses (()) used in the select attribute are essential to correct behavior. The predicate [1] is the same as [position()=1].

Checking the position in the second template rather than in the first moves the decision from the caller of the template to the template itself, as in Listing 26.

<xsl:template match="Documents">
  <xsl:apply-templates select="//Name"/>
</xsl:template>

<xsl:template match="Name">
  <xsl:if test="position()=1">
    <xsl:value-of select="concat(position(),' ',.,'&#10;')"/>
  </xsl:if>
</xsl:template>

The check for position has been moved to within the template itself. This check is part of an xsl:if statement rather than a predicate in the match expression. If the predicate is in the template's match expression—that is, <xsl:template match="Name[1]">—the results are those in Listing 23. The results of Listing 26 are given in Listing 27.

1 Joe Bloggs

Listing 28 includes a template to ignore white space text nodes that affect the position of the <Name> elements in Listing 17.

<xsl:template match="*">
  <xsl:apply-templates select="*"/>
</xsl:template>

<xsl:template match="Name">
  <xsl:value-of select="concat(position(),' ',.,'&#10;')"/>
</xsl:template>

Listing 29 shows the results with the text nodes ignored.

1 Knowing the Good
1 Bob Bloggs
2 Kim Bergess
1 Beyond Standards: Best Practices
1 Bill Agnew
1 101 Ways to Do the Same Thing
1 Jay Nerks
2 Paula Towne
3 Carol Tinney

Different results are returned by the position() function depending on where it's used and whether it's in a template's match expression, in a predicate, or in the selected node set of an xsl:apply-templates.

Back to top

Tip 4: Selecting the first element with a certain name

You can select the first element in an XML document with the name Name using either //Name[not(preceding::Name)], which is required for a template match, or (//Name)[1], if used in a select attribute. The easiest method for selecting the first in a set of elements is with a predicate of value 1. For example, Listing 30 is an XML document of documents with name, year published, and number of pages.

<Documents>
  <Document id="18396">
    <Name>Knowing the Good</Name>
    <Year>2010</Year>
    <Pages>35</Pages>
  </Document>
  <Document id="18397">
    <Name>Beyond Standards: Best Practices</Name>
    <Year>2011</Year>
    <Pages>12</Pages>
  </Document>
  <Document id="18398">
    <Name>101 Ways to Do the Same Thing</Name>
    <Year>2011</Year>
    <Pages>50</Pages>
  </Document>
</Documents>

The XPath expression //Document[1]—or, alternately, /Documents/Document[1]—selects the first document element. The result is Listing 31.

<Document id="18396">
  <Name>Knowing the Good</Name>
  <Year>2010</Year>
  <Pages>35</Pages>
</Document>

The XPath expression selects the list of <Document> elements and then returns the first one. The predicate [1] means "the element that has a position of 1." Another way of expressing the same thing is //Document[position()=1].

Next, consider how to select the name of the first document. The <Name> element is a child node of the <Document> element. Select the child by appending its tag name to the XPath—that is, //Document[1]/Name, which returns the <Name> element of the first <Document> element. The result is Listing 32.

<Name>Knowing the Good</Name>

The XPath expression //Name[1] selects the first <Name> element regardless of where it's located in the XML document; however, there is a problem. The unexpected result is Listing 33. Three elements are returned instead of just one.

<Name>Knowing the Good</Name>
<Name>Beyond Standards: Best Practices</Name>
<Name>101 Ways to Do the Same Thing</Name>

Three elements are returned when only one is expected. The first <Name> element in each <Document> element is selected. If a <Document> element contained more than one <Name> element, only the first one would be returned.

How you select the first <Name> element in the XML document depends on how you use XPath. In an XSLT template match attribute, the options are more limited than with the select attribute. For a match XPath expression, use the XPath axis preceding to choose the first one. Listing 34 illustrates the expression.

<xsl:template match="//Name[not(preceding::Name)]">

The expected result, the first element, is Listing 35.

<Name>Knowing the Good</Name>

The expression means "select the <Name> element that has no <Name> elements earlier in the XML document."

For a select expression, such as xsl:apply-templates statements, the XPath is more flexible. Although the expression in Listing 34 works, using carefully applied parentheses does the job more cleanly and is easily adapted to select the second, third, or any other index. See Listing 36.

<xsl:apply-templates select="(//Name)[1]"/>

The result is Listing 37, which is the same as Listing 35.

<Name>Knowing the Good</Name>

The expression selects all the <Name> elements, then returns the first one.

Careful placement of parentheses and axes is important when selecting the desired results.

Back to top

Tip 5: Handling XPath select expressions that fail to match documents with a default namespace

The default namespace of an XSLT does not apply to its XPath expressions. Three possible solutions exist:

• Delete the XML document's default namespace.
• Add a namespace prefix to XPath expressions.
• Preprocess to remove the namespace.

Option 1: Delete the XML document's default namespace

If the XML document can be modified, delete the default namespace. Listing 38 is an XHTML document—by definition, an XML document—with a default namespace.

<?xml version="1.0"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
    "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
  <head>
    <title>This is the title</title>
  </head>
  <body>
    <p>Hello world.</p>
  </body>
</html>

Option 2: Add a namespace prefix to XPath expressions

Listing 39 is an XSLT in which the XML document's default namespace is assigned a prefix. The prefix xhtml was chosen for the example, but any prefix is acceptable.

<?xml version="1.0"?>
<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform" 
    xmlns:xhtml="http://www.w3.org/1999/xhtml" 
    xmlns="http://www.w3.org/1999/xhtml" 
    exclude-result-prefixes="xhtml" >
  <xsl:output method="xml" version="1.0"/>
  <xsl:template match="xhtml:title">
    <title>Replaced with a new title</title>
  </xsl:template>

  <xsl:template match="@*|node()">
    <xsl:copy>
      <xsl:apply-templates select="@*|node()"/>
    </xsl:copy>
  </xsl:template>
</xsl:stylesheet>

Listing 40 shows the resulting XHTML document. The text in the <title> element has been correctly replaced. The xhtml: prefix is excluded from the transformed output document by using the exclude-result-prefixes attribute in the XSLT.

<?xml version="1.0"?>
<html xmlns="http://www.w3.org/1999/xhtml">
  <head>
    <title>Replaced with a new title</title>
  </head>
  <body>
    <p>Hello world.</p>
  </body>
</html>

Option 3: Preprocess to remove the namespace

Removing the namespace from the XML document by preprocessing it with an XSLT might be an option, but doing so might be time-consuming. The XSLT in Listing 41 copies text, element, and attribute nodes in the source XML document, but namespace nodes and the DOCTYPE declaration are omitted.

<?xml version="1.0"?>
<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
  <xsl:output method="xml" version="1.0"/>

  <!-- copy elements without namespace -->
  <xsl:template match="*">
    <xsl:element name="{name()}">
      <xsl:apply-templates select="node()|@*"/>
    </xsl:element>
  </xsl:template>

  <xsl:template match="@*|text()|processing-instruction()|comment()">
    <xsl:copy>
      <xsl:apply-templates select="@*|node()"/>
    </xsl:copy>
  </xsl:template>
</xsl:stylesheet>

This two-pass approach transforms the document twice, with the result of the first XSLT transformed by a second XSLT. For more on multi-pass transforms, see Resources.

Option 1, if you can apply it, is the easiest and simplest solution. Option 2 is possible if the XML documents consistently have a default namespace, which is usually the case. Although a little extra typing is required to add the namespace prefixes to the XPath expressions, this solution works and doesn't add extra processing. Option 3 processes the XML document twice and is, therefore, slower. Concerns about whether the source XML document has a default namespace, however, are eliminated.

Back to top

Conclusion

Binary logic in XPath expressions is a bit more complex than simply true and false. The absence of a value resolves to False when comparing for equality or inequality. Conversion of Boolean values to strings, when interpreted as a Boolean, is evaluated as True. The third tip explained the relative nature of the position() function depending on its context, and the fourth tip showed how to use the XPath preceding axis or a predicate to select the first element of a given name in a list. Finally, the common problem of failing XPath select expressions was identified as the result of a default namespace in the XML document, and several possible solutions were given.

XPath is a powerful functional language for selecting nodes in an XML document. But as with all computer languages, XPath comes with a learning curve. Experiment with these examples to gain an even deeper understanding.

Resources

Learn

• XSL Transformations (XSLT) Version 1.0 (W3C Recommendation, November 1999 ): Read the specification that defines the syntax and semantics of XSLT, which is a language for transforming XML documents into other XML documents.
  
  
• Tip: Multi-pass XSLT: Use the node-set extension to break down the XSLT operation into two or more passes (Uche Ogbuji, developerWorks, September 2002): Learn how to transform a document in preparation for a main transformation.
  
  
• The Java XPath API (Elliotte Rusty Harold, developerWorks, August 2008): Find more on using XPath with the Java™ API in this introduction of the javax.xml.xpath package, an XML object model-independent library for querying documents with XPath.
  
  
• Remember the difference between value1!= value2 and not(value1 = value2): Check out another example of distinctive logic among nine other helpful tips on XPath in the fourth point of "Ten Tips to Using XPath and XPointer."
  
  
• XPath axes: Learn more about these node-set relatives to the current node from W3schools.com.
  
  
• New to XML? Get the resources you need to learn XML.
  
  
• More articles by this author (Doug Domeny, developerWorks, December 2010-current): Read articles about XSLT, XHTML, and other technologies.
  
  
• XML area on developerWorks: Get the resources you need to advance your skills in the XML arena. See the XML technical library for a wide range of technical articles and tips, tutorials, standards, and IBM Redbooks
  
  
• IBM XML certification: Find out how you can become an IBM-Certified Developer in XML and related technologies.
  
  
• developerWorks technical events and webcasts: Stay current with technology in these sessions.
  
  
• developerWorks on Twitter: Join today to follow developerWorks tweets.
  
  
• developerWorks podcasts: Listen to interesting interviews and discussions for software developers.
  
  
• developerWorks on-demand demos: Watch demos ranging from product installation and setup for beginners to advanced functionality for experienced developers.
  
  

Get products and technologies

• IBM product evaluation versions: Download or explore the online trials in the IBM SOA Sandbox and get your hands on application development tools and middleware products from DB2®, Lotus®, Rational®, Tivoli®, and WebSphere®.
  
  

Discuss

• XML zone discussion forums: Participate in any of several XML-related discussions.
  
  
• The developerWorks community: Connect with other developerWorks users while exploring the developer-driven blogs, forums, groups, and wikis.
  
  

About the author

Rate this article

Comments

Back to top

Table of contents

• Tip 1: False is sometimes true
• Tip 2: Know the difference between (x != y) and not(x = y)
• Tip 3: The value of position() depends on where it is used
• Tip 4: Selecting the first element with a certain name
• Tip 5: Handling XPath select expressions that fail to match documents with a default namespace
• Conclusion
• Resources
• About the author
• Comments

Local resources

• IBM Innovation Center - Austin, TX
• IBM Innovation Center - Chicago, IL
• IBM Innovation Center - Dallas, TX
• IBM Innovation Center - San Mateo, CA
• IBM Innovation Center - Waltham, MA
• Technical Briefings in North America

Dig deeper into XML on developerWorks

Try IBM PureSystems. No charge.

Experience today!

---

Get started with the cloud-based trial and pattern development kit.

Special offers