BeautifulSoup: object of type 'Response' has no len()

问题

Issue: when I try to execute the script, BeautifulSoup(html, ...) gives the error message "TypeError: object of type 'Response' has no len(). I tried passing the actual html as a parameter, but it still doesn't work.

import requests

url = 'http://vineoftheday.com/?order_by=rating'
response = requests.get(url)
html = response.content

soup = BeautifulSoup(html, "html.parser")

回答1:

You are getting response.content. But it return response body as bytes (docs). But you should pass str to BeautifulSoup constructor (docs). So you need to use the response.text instead of getting content.

回答2:

Try to pass the HTML text directly

soup = BeautifulSoup(html.text)

回答3:

If you're using requests.get('https://example.com') to get the HTML, you should use requests.get('https://example.com').text.

回答4:

you are getting only response code in 'response' and always use browser header for security otherwise you will face many issues

Find header in debugger console network section 'header' UserAgent

Try

import requests
from bs4 import BeautifulSoup

from fake_useragent import UserAgent

url = 'http://www.google.com'
headers = {'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) 
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36'}

response = requests.get(quote_page, headers=headers).text

soup = BeautifulSoup(response, 'html.parser')
print(soup.prettify())

回答5:

It worked for me:

soup = BeautifulSoup(requests.get("your_url").text)

Now, this code below is better (with lxml parser):

import requests
from bs4 import BeautifulSoup

soup = BeautifulSoup(requests.get("your_url").text, 'lxml')

来源：https://stackoverflow.com/questions/36709165/beautifulsoup-object-of-type-response-has-no-len

标签

python

html

parsing

web-scraping

beautifulsoup