Python selenium refresh if wait more than 10s

Question

from selenium import webdriver    
driver=webdriver.Firefox() 
driver.get(url)

Sometimes the webdriver is stuck on a file or response and the page is never full-loaded so the line

driver.get(url) 

is never finished. But I already got enough source code to run the rest of my code. I am wondering how can I bypass or refresh the page if the page is not full-loaded in 10 seconds.

I have tried

from selenium import webdriver
from selenium.common.exceptions import TimeoutException    
driver=webdriver.Firefox() 
driver.set_page_load_timeout(10)
while True:
    try:
        driver.get(url)
    except TimeoutException:
        print("Timeout, retrying...")
        continue
    else:
        break

but the line

driver.set_page_load_timeout(10)

always gives me following error

  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/selenium/webdriver/remote/webdriver.py", line 727, in set_page_load_timeout
'pageLoad': int(float(time_to_wait) * 1000)})
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/selenium/webdriver/remote/webdriver.py", line 238, in execute
self.error_handler.check_response(response)
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/selenium/webdriver/remote/errorhandler.py", line 193, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: 

This is nothing after Message:. I can't identify the type of error. It's weird that my laptop can't run

driver.set_page_load_timeout(10)

My next step is to click a button on the page, but that button doesn't always exist even after full-loaded. Thus I can't use explicit wait.

Thanks

Answer 1

(In your code snippet you don't define URL, but I'll assume URL is defined somewhere in your actual code.)

You could combine the retry and timeout-decorator packages for this:

from retry import retry
from timeout_decorator import timeout, TimeoutError
from selenium import webdriver
from selenium.common.exceptions import TimeoutException    

@retry(TimeoutError, tries=3)
@timeout(10)
def get_with_retry(driver, url):
    driver.get(url)


def main():
    url = "http://something.foo"

    driver=webdriver.Firefox() 
    try:
        get_with_retry(driver, url)
        foo(driver) # do whatever it is you need to do
    finally:
        driver.quit()


if __name__ == "__main__":
    main()

Note that you would need to either not set driver.set_page_load_timeout to anything, or set it to something higher than 10 seconds.