Python - Urllib2 Wait for page to load to scrape data

无人久伴 提交于 2019-12-20 04:09:39

问题


Firstly, I'd like to say that I do not want to use any libraries that are not provided with Python 2.7.10. The same question was posted on Stack Overflow but was answered with the Requests library.

I have a script that logs into Roblox.com using urllib2. To check if there is a captcha before I try to log in, I wanted to do check_captcha = re.findall('recaptcha_image', newlogin) but roblox needs to redirect to the captcha login page AND the captcha has to load onto the page.

So how can I make Python wait to redirect/load the page fully before I go ahead and .read() and scrape it.


回答1:


This will wait 10 seconds before it reads it:

import urllib2
import time
url = 'Roblox url'
data = urllib2.urlopen(url)
time.sleep(10)
data = data.read()


来源:https://stackoverflow.com/questions/31310321/python-urllib2-wait-for-page-to-load-to-scrape-data

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!