Python BeautifulSoup equivalent to lxml make_links_absolute

喜欢而已 提交于 2019-12-04 21:55:59

问题


So lxml has a very hand feature: make_links_absolute:

doc = lxml.html.fromstring(some_html_page)
doc.make_links_absolute(url_for_some_html_page)

and all the links in doc are absolute now. Is there an easy equivalent in BeautifulSoup or do I simply need to pass it through urlparse and normalize it:

soup = BeautifulSoup(some_html_page)
for tag in soup.findAll('a', href=True):
    url_data = urlparse(tag['href'])
    if url_data[0] == "":
        full_url = url_for_some_html_page + test_url

回答1:


In my answer to What is a simple way to extract the list of URLs on a webpage using python? I covered that incidentally as part of the extraction step; you could easily write a method to do it on the soup and not just extract it.

import urlparse

def make_links_absolute(soup, url):
    for tag in soup.findAll('a', href=True):
        tag['href'] = urlparse.urljoin(url, tag['href'])


来源:https://stackoverflow.com/questions/4468410/python-beautifulsoup-equivalent-to-lxml-make-links-absolute

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!