Using xmllint and xpath with a less-than-perfect HTML document?

生来就可爱ヽ(ⅴ<●) 提交于 2019-12-04 10:50:49

问题


I have an HTML page that is generated by an existing tool - I cannot change the output of this tool.

However, I want to use xmllint with the --xpath option to pick out a few specific pieces of information from the downloaded webpage. The problem is that the page starts with:

<html lang=en><head>...

And xmllint throws errors nearly immediately:

html.out:2: parser error : AttValue: " or ' expected
<html lang=en><head>
           ^

The issue certainly seems to be the missing enclosing quotation marks around the value of the lang attribute. The entire page is full of this kind of issue. (Though only sporadically.)

Nearly every browser can parse this just fine - how can I convince xmllint to do so as well? I would like to avoid having to inject an intermediate step to "fix" the file. Instead, I would like to either:

1) Find a flag, validation option, etc. that helps the parser along, or:

2) Use some other tool. (But what? xmllint is always my go-to for command line XPath commands.)

Further, using just xpath results in:

> xpath html.out '//myquery...'

not well-formed (invalid token) at line 2, column 11, ...

回答1:


You can enable the HTML parser in xmllint using the --html command line option. That way, you will be able to process HTML documents.




回答2:


You should pre-process the HTML with a lenient parser. (That's the main difference: HTML is allowed a much more lax syntax than XML.) That is, try HTML5-Tidy and let XMLLint work on the result:

input HTML
 |
 v
Tidy
 |
 v
xmllint
 |
 v
result



回答3:


If does not abort the parsing, you can just hide the errors with:

2>/dev/null

Then there is Xidel, which I made just for picking some data from html pages. (although it is not perfect. I was told about two malformed documents it could not handle)

xidel  html.out -e //yourquery...


来源:https://stackoverflow.com/questions/21479855/using-xmllint-and-xpath-with-a-less-than-perfect-html-document

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!