Web scrapping with Python

问题

How to parse table from https://ege.hse.ru/rating/2019/81031971/all/?rlist=&ptype=0&vuz-abiturients-budget-order=ge&vuz-abiturients-budget-val=10 with BeautifulSoup and make pandas DataFrame? My code:

import requests
from bs4 import BeautifulSoup
url = 'https://ege.hse.ru/rating/2019/81031971/all/?rlist=&ptype=0&vuz-abiturients-budget-order=ge&vuz-abiturients-budget-val=10'
page = requests.get(url)
soup = BeautifulSoup(page.content,"html.parser")
table = soup.find_all("table")
for each_table in table:
 for row in each_table.find_all('tr'):
  for cell in row.find_all("td"):
   print(cell.text)

I try this:

import requests
from bs4 import BeautifulSoup
import pandas as pd

url = "https://ege.hse.ru/rating/2019/81031971/all/?rlist=&ptype=0&vuz-abiturients-budget-order=ge&vuz-abiturients-budget-val=10"
page = requests.get(url)

soup = BeautifulSoup(page.content, 'html.parser')

tbl = soup.find("table", {"id": "MainContent_dataGridView1"})

data_frame = pd.read_html(str(tbl))[0]
print(data_frame)

But it says:

"ValueError: No tables found"

回答1:

I see only a table with id="transparence_t"

So:

tbl = soup.find("table", {"id": "transparence_t"})
data_frame = pd.read_html(str(tbl))[0]
print(data_frame)

It returns to me a 698x6 dataframe

来源：https://stackoverflow.com/questions/59181538/web-scrapping-with-python