How to find XML Elements via XPath in Python in a namespace-agnostic way?

Question

since I had this annoying issue for the 2nd time, I thought that asking would help.

Sometimes I have to get Elements from XML documents, but the ways to do this are

Answer 1

The *[local-name() = "elem"] syntax should work, but to make it easier you can create a function to simplify construction of the partial or full "wildcard namespace" XPath expressions.

I'm using python-lxml 2.2.4 on Ubuntu 10.04 and the script below works for me. You'll need to customize the behavior depending on how you want to specify the default namespaces for each element, plus handle any other XPath syntax you want to fold into the expression:

import lxml.etree

def xpath_ns(tree, expr):
    "Parse a simple expression and prepend namespace wildcards where unspecified."
    qual = lambda n: n if not n or ':' in n else '*[local-name() = "%s"]' % n
    expr = '/'.join(qual(n) for n in expr.split('/'))
    nsmap = dict((k, v) for k, v in tree.nsmap.items() if k)
    return tree.xpath(expr, namespaces=nsmap)

doc = '''
    
'''

tree = lxml.etree.fromstring(doc)
print xpath_ns(tree, '/root')
print xpath_ns(tree, '/root/elem')
print xpath_ns(tree, '/root/other:elem')

Output:

[]
[]
[]

Update: If you find out you do need to parse XPaths, you can check out projects like py-dom-xpath which is a pure Python implementation of (most of) XPath 1.0. In the least that will give you some idea of the complexity of parsing XPath.