问题
We're using Beautiful Soup to parse many websites successfully, but a few are given us problems. An example is this page:
http://www.designsponge.com/2013/04/biz-ladies-how-to-use-networking-to-improve-your-search-engine-rankings.html
We're feeding the exact source to beautiful soup, but it returns a stunted HTML string, though no errors...
Code:
soup = BeautifulSoup(site_html)
print str(soup.html)
Result:
<html class="no-js" lang="en"> <!--<![endif]--> </html>
I'm trying to determine what's tripping it up, but nothing jumps out at me looking at the html source. Does anyone have some insight?
回答1:
Try different parsers, the page parses fine with the html5lib parser:
>>> soup = BeautifulSoup(r.content, 'html5')
>>> len(soup.find_all('li'))
97
Not all parsers can treat broken HTML the same.
来源：https://stackoverflow.com/questions/15773977/beautiful-soup-failing-to-parse-this-html