Parsing the result obtained from mochiweb_html

问题

I would like to parse some content from an html file (no xml).

At the moment I retrieve the structure to parse using mochiweb_html:

1> inets:start().
2> {ok, {Status, Headers, Body}} = httpc:request("http://www.google.com").
3> {String, Attributes, Other} = mochiweb_html:parse(Body).

and the result is something like:

{<<"html">>,
 [{<<"itemscope">>,<<"itemscope">>},
  {<<"itemtype">>,<<"http://schema.org/WebPage">>}],
 [{<<"head">>,[],
   [{<<"meta">>,
     [{<<"itemprop">>,<<"image">>},
      {<<"content">>,<<"/images/google_favicon_128.png">>}],
     []},
    {<<"title">>,[],[<<"Google">>]},
....

What is the best way to retrieve from a structure obtained from mochiweb_http all the elements in the web page that have a specific tag with a specific class (e.g., <span id="footer">)?

回答1:

You could use mochiweb_xpath:

> mochiweb_xpath:execute("//span[@id='footer']",
    mochiweb_html:parse(
      "<html><body><span>not this one</span><span id='footer'>but this one</span></body></html>")).
[{<<"span">>,
  [{<<"id">>,<<"footer">>}],
  [<<"but this one">>]}]

回答2:

It depends on your performance requirements.

The mochiweb result is in a 3-tuple form that could probably fairly easily be converted to input suitable for xmerl. Most of the work would be converting attribute names to atoms. Then you could use xmerl_xpath to do some pretty flexible querying.

Otherwise you could code something less flexible (but maybe faster) to walk the tree.

来源：https://stackoverflow.com/questions/16148202/parsing-the-result-obtained-from-mochiweb-html

标签

erlang

mochiweb