Dynamic Data Web Scraping with Python, BeautifulSoup

Question

I am trying to extract this data(number) for many pages from the HTML. The data is different for each page. When I try to use soup.select(\'span[class=\"pull-right\"]\') it

Answer 1

You should try using the python plugin selenium. It requires you to download a driver for whatever browser you are using. You will then be able to use selenium to pull out values from the html

from selenium import webdriver
from selenium.webdriver.common.keys import Keys

driver = webdriver.Firefox()
driver.get("https://tools.wmflabs.org/pageviews/?project=en.wikipedia.org&platform=all-access&agent=user&range=latest-20&pages=Star_Wars:_The_Last_Jedi")
element = driver.find_element_by_class_name("pull-right")
// or the following below 
//element = driver.find_element_by_name("q")
//element = driver.find_element_by_id("html ID name")
//element = driver.find_element_by_name("html element name")
//element = driver.find_element_by_xpath("//input[@id='passwd-id']")
print(element)
driver.close()