问题
I am trying to extract table data from this web site
Following is the code--
import requests
from bs4 import BeautifulSoup as bs
page = requests.get('https://www.vitalityservicing.com/serviceapi/Monitoring/QueueDepth?tenantId=1')
soup = bs(page.text, "html.parser")
#None of the following method works
tb = soup.table
#tb = soup.body.table
#tb = soup.find_all('table')
When I try to print tb its None
So I tried to look at the body of the downloaded HTML with
print(soup.body.prettify())
I dont see the table elements or it's child elements . Only <body> and <script> elements are present:
But when I inspect the page in chrome I see all the elements:
I don't understand why the table element is not being downloaded with requests.get while its there when I load the page on chrome
回答1:
You are not getting that content because, when you execute the request, it is not present in the page. Yet.
If you check the javascript code between the script tags, you can see it is generating the table dynamically. So, you receive the html code before that has happened, since requests is not a browser and won't execute the js, and you don't get to see the table.
Now that you know why you can't see the table, your next problem is how to get the HTML produced after the javascript execution. Don't faint, it is feasible. You might find the solutions in this question interesting.
Good luck
来源：https://stackoverflow.com/questions/51897756/table-element-not-showing-in-beautifulsoup