How can I scroll a web page using selenium webdriver in python?

前端 未结 18 1966
孤街浪徒
孤街浪徒 2020-11-22 07:04

I am currently using selenium webdriver to parse through facebook user friends page and extract all ids from the AJAX script. But I need to scroll down to get all the friend

18条回答
  •  暗喜
    暗喜 (楼主)
    2020-11-22 07:32

    scroll loading pages. Example: medium, quora,etc

    last_height = driver.execute_script("return document.body.scrollHeight")
        while True:
            driver.execute_script("window.scrollTo(0, document.body.scrollHeight-1000);")
            # Wait to load the page.
            driver.implicitly_wait(30) # seconds
            new_height = driver.execute_script("return document.body.scrollHeight")
        
            if new_height == last_height:
                break
            last_height = new_height
            # sleep for 30s
            driver.implicitly_wait(30) # seconds
        driver.quit()
    

提交回复
热议问题