How to scroll down in Python Selenium step by step

问题

Hi guys I am new to Selenium and Python. I was just scraping the site pagalguy website. I know how to scroll down to the bottom of the page but what I need is to scroll down step by step so that the Selenium will click all the readmore buttons,but I don't know how to scroll down step by step like that so I hard coded it like following one

browser.execute_script("window.scrollTo(0,300);")
browser.find_element_by_link_text("Read More...").click()

browser.execute_script("window.scrollTo(300,600);")
browser.find_element_by_link_text("Read More...").click()

browser.execute_script("window.scrollTo(600,900);")
browser.find_element_by_link_text("Read More...").click()

browser.execute_script("window.scrollTo(900,1200);")
browser.find_element_by_link_text("Read More...").click()

browser.execute_script("window.scrollTo(1200,1500);")
browser.find_element_by_link_text("Read More...").click()

browser.execute_script("window.scrollTo(1500,1800);")
browser.find_element_by_link_text("Read More...").click()

browser.execute_script("window.scrollTo(1800,2100);")
browser.find_element_by_link_text("Read More...").click()

browser.execute_script("window.scrollTo(2100,2500);")
browser.find_element_by_link_text("Read More...").click()
it goes on .......

I tried automating it using a while loop but it resulted in error, the above one works but I want it short and looped so that I can use it for all the other pages with different page length.

initial_value = 0
next_value = 300
while next_value<300000: 
  browser.execute_script("window.scrollTo(initial_value,next_value);")
  browser.find_element_by_link_text("Read More...").click()
  initial_value=next_value
  next_value+=300

JavascriptException: Message: ReferenceError: initial_value is not defined

But I have defined the value, I think I explained what I am actually trying to do, I want to automatically scroll down and click all the readmore buttons and then I will get the full text content

回答1:

Agree with answer of @Rahul Chawla.

But adding one change. You can try this one

driver = webdriver.Chrome()

read_mores = driver.find_elements_by_xpath('//a[text()="Read More..."]')

for read_more in read_mores:
    driver.execute_script("arguments[0].scrollIntoView();", read_more)
    driver.execute_script("$(arguments[0]).click();", read_more)

回答2:

We can do this by finding all read more buttons using find_elements_by_xpath() and looping over them while scrolling them into view one by one.

driver = webdriver.Chrome()

read_mores = driver.find_elements_by_xpath('//a[text()="Read More..."]')

for read_more in read_mores:
    driver.execute_script("arguments[0].scrollIntoView();", read_more)
    read_more.click()
    # your code here

回答3:

Use loop with javascript window.scrollBy(0, Y) method with coordinates selecting movement step and iterations number.

   for i in range(20): # adjust integer value for need
       # you can change right side number for scroll convenience or destination 
       driver.execute_script("window.scrollBy(0, 250)")
       # you can change time integer to float or remove
       time.sleep(1)

回答4:

Try like this:

while next_value<300000:
    driver.execute_script("window.scrollTo({},{});".format(initial_value, next_value))
    browser.find_element_by_link_text("Read More...").click()
    initial_value=next_value
    next_value+=300

Basically I just changed this line: browser.execute_script("window.scrollTo(initial_value,next_value);")

for this one:

driver.execute_script("window.scrollTo({},{});".format(initial_value, next_value))

来源：https://stackoverflow.com/questions/48006078/how-to-scroll-down-in-python-selenium-step-by-step