# XPointer processing

## About XPointer

XPointer is a set of W3C specifications (one part of which is, as of 2006, still a Working Draft) that provide a means of identifying and referring to a portion of an XML document. The portion being referenced need not be contiguous, and need not constitute a well-formed general entity. XPointers were originally intended to be used in the fragment component of a URI or IRI (the fragment being the part after "`#`"), but the specifications actually place no restrictions on where they can be used.

An example of an XPointer embedded in a URI would be

`http://example.com/inventory.xml#xpointer(//part%5Bstarts-with(sku,%20'999')%5D)`

The XPointer in that example is actually

`xpointer(//part[starts-with(sku, '999')])`

but the URI syntax requires further encoding of some data. The result of evaluating this XPointer would be the same as evaluating the XPath expression `//part[starts-with(sku, '999')]` against the document identified by the URI `http://example.com/inventory.xml`.

XPointer syntax is simple: a shorthand XPointer is just a name, and refers to the element with that ID (as determined by a DTD or other schema, typically), much like the XPath 1.0 expression `id(somename)`, but with a little more flexibility, since `id()` is limited to DTD-based data typing.

A scheme-based XPointer consists of a series of one or more parts, separated by optional whitespace, with each part looking like a function call. What appear to be function names are actually syntactic and semantic schemes, of which the most common is the ID-oriented `element` scheme, and of which the most versatile is the XPath-oriented `xpointer` scheme.

If a scheme-based XPointer contains more than one part, then the parts are evaluated from left to right, skipping any unsupported/unrecognized schemes, until one is found that identifies something that exists in the document. Some schemes, like the namespace/prefix-binding `xmlns`, identify nothing (by design), and instead just influence the interpretation of subsequent parts. It's possible for an XPointer to produce different results with different processors, if the author doesn't take care to ensure each part identifies the same thing.

Here are some more examples:

The XPath 1.0 expression `id(somename)` means the same thing as the XPointer `xpointer(id(somename))`, and nearly the same thing as the XPointers `element(somename)` and `somename`, which just have more flexibility in where the ID can be drawn from.

The XPointer `element(somename/3/1)` means nearly the same thing as the XPath expression `id(somename)/*[3]/*[1]`.

The XPointer `xmlns(xhtml=http://www.w3.org/1999/xhtml)xpointer(//xhtml:a[@href])` could be used to refer to all of the links in an XHTML 1.0 document.

## XPointer support in 4Suite

4Suite's XPointer implementation, sometimes called 4XPointer, has no command-line interface, but can be used within Python applications. It supports XPointers to different degrees, depending on the circumstances:

1. When an XML document is being parsed into a Domlette with XInclude processing enabled, any XPointer encountered in an `xi:include` element is automatically evaluated when the included document is parsed. In this mode the XPointer must use an XPath LocationPath that only uses steps along the child axis. Furthermore, any predicates must be literal numbers, or must be of the specific form `[@attname='attvalue']`. For example, `/foo[3]` and `/foo[@bar='baz']` will work, but `../foo` and `foo/[.='bar']` will not. Function calls are not allowed.

2. If you have not yet parsed an XML document, but have a URI for it, then you can use Ft.Xml.XPointer.SelectUri() to parse the document and evaluate an XPointer embedded in the URI's fragment component. The parsing is performed with Domlette's default NonvalidatingReader instance. There are some implementation gaps to note when using the `xpointer` scheme: the only additional function fully supported is here(), and the following functions always return empty location-sets: string-range(), range-to(), start-point(), end-point(), and origin(). origin is illegal to use outside of extended XLinks, anyway.

3. If you have already parsed the document into a Domlette, then you can evaluate an arbitrary XPointer against it by using Ft.Xml.XPointer.SelectNode(). The same implementation gaps as noted in the description of Ft.Xml.XPointer.SelectUri() apply.

Ranges are not supported because Domlette does not support DOM Level 2 Ranges. Uche Ogbuji posted some thoughts about this topic a while back. Also note that although the `element` scheme is streamable, it is not yet supported in XIncludes due to ID-related limitations in Domlette. Since `element` and shorthand pointer support are requirements for full XInclude conformance, they will probably be implemented in the future.

In 4Suite 1.0b1 and earlier, the implementation was based on older versions of the specs, and several additional restrictions were in effect: the `element` scheme was not even an option, XPointers in XIncludes had to be given via URIs (not attributes) and couldn't contain NameTests involving "`*`", and all other XPointers were only allowed to identify a single node.

## Examples

The following example uses XInclude with XPointer references to include various sections from one document into another document. lists the contents of the file `article.xml`, which references one section using a shorthand pointer and then references any sections that have their condition attribute set to unfinished. lists the contents of the file `article2.xml`, which is referenced from `article.xml`. lists the Python code used to parse and print this document; note that XPointer processing is driven from XInclude processing, which is done automatically by default.

`article.xml`: Document using XInclude with XPointer references

    <article>
      <title>My important article</title>
      <xi:include href="article2.xml"
                  xpointer="woo"
                  xmlns:xi="http://www.w3.org/2001/XInclude"/>
      <xi:include href="article2.xml"
                  xpointer="xpointer(article/section[@condition='unfinished'])"
                  xmlns:xi="http://www.w3.org/2001/XInclude"/>
    </article>

`article2.xml`: Document with content referenced from `article.xml`

    <article>
      <section condition="unfinished">
        <title>Section 1</title>
        <!-- Write me! -->
      </section>
      <section xml:id="woo">
        <title>Section 2</title>
        <para>Yeah, content.</para>
      </section>
      <section condition="unfinished">
        <title>Section 3</title>
        <!-- Write me, too! -->
      </section>
    </article>

Loading the document

    from Ft.Xml import Parse
    from Ft.Xml.Domlette import PrettyPrint
    doc = Parse("article.xml")
    PrettyPrint(doc)

As mentioned earlier, XPointer is most commonly used along with XInclude, but 4Suite provides an API for using XPointer directly from Python. Using `article2.xml` as listed above in , loads two of the nodes loaded previously with XInclude. Note that when using the standalone interface, the code is able to take advantage of more of the XPointer syntax.

Using XPointer directly from Python

    from Ft.Xml import Parse
    from Ft.Xml.Domlette import PrettyPrint
    from Ft.Xml.XPointer import SelectNode

    article2 = Parse("article2.xml")
    # Shorthand XPointer syntax
    node = SelectNode(article2, "woo")[0]
    PrettyPrint(node)
    # Scheme-based XPointer syntax
    node = SelectNode(article2,
                      "xpointer(//section[@condition='unfinished'][2])")[0]
    PrettyPrint(node)

is very similar to the examples above, only this version is self-contained; the resources are stored in Python strings and resolved using a custom resolver.

Self-contained example

    article = """<article><title>My important article</title>
    <xi:include href="ex:article2"
                xpointer="woo"
                xmlns:xi="http://www.w3.org/2001/XInclude"/>
    <xi:include href="ex:article2"
                xpointer="xpointer(article/section[@condition='unfinished'])"
                xmlns:xi="http://www.w3.org/2001/XInclude"/>
    </article>"""

    article2 = """<article>
    <section condition="unfinished"><title>Section 1</title><!-- Write me! --></section>
    <section xml:id="woo"><title>Section 2</title><para>Yeah, content.</para></section>
    <section condition="unfinished"><title>Section 3</title><!-- Write me, too! --></section>
    </article>"""

    from Ft.Lib.Uri import FtUriResolver, Absolutize
    from Ft.Lib import UriException
    from cStringIO import StringIO
    class MyResolver (FtUriResolver):
      def normalize(self, uriRef, baseUri):
        return Absolutize(uriRef, baseUri)
      def resolve(self, uri):
        if uri == "ex:article":
          return StringIO(article)
        elif uri == "ex:article2":
          return StringIO(article2)
        else:
          raise UriException(UriException.RESOURCE_ERROR,
                             loc=uri, msg="not found, sorry")

    myResolver = MyResolver()

    from Ft.Xml.InputSource import InputSourceFactory
    from Ft.Xml.Domlette import NonvalidatingReader, PrettyPrint
    factory = InputSourceFactory(resolver=myResolver)
    isrc = factory.fromUri("ex:article")
    doc = NonvalidatingReader.parse(isrc)
    PrettyPrint(doc)

    from Ft.Xml.XPointer import SelectNode

    isrc = factory.fromUri("ex:article2")
    article2 = NonvalidatingReader.parse(isrc)
    node = SelectNode(article2, "woo")[0]
    PrettyPrint(node)
    node = SelectNode(article2,
                      "xpointer(//section[@condition='unfinished'][2])")[0]
    PrettyPrint(node)