urllib2

Permanent 'Temporary failure in name resolution' after running for a number of hours

由 被刻印的时光 ゝ 提交于 2019-12-03 17:57:45
问题 After running for a number of hours on Linux, my Python 2.6 program that uses urllib2, httplib and threads, starts raising this error for every request: <class 'urllib2.URLError'> URLError(gaierror(-3, 'Temporary failure in name resolution'),) If I restart the program it starts working again. My guess is some kind of resource exhaustion but I don't know how to check for it. How do I diagnose and fix the problem? 回答1: This was caused by a library's failure to close connections, leading to a

Python - Getting url a browser was redirected to

由 梦想的初衷 提交于 2019-12-03 15:14:36
问题 I am trying to authenticate an application with the API. Here's How: I am opening a URL using webbrowser.open . The user authenticates the application, and is redirected to another URL, which is https://stackexchange.com/oauth/login_success with arguments encoded with this URL. A sample redirect url is: .../login_success#access_token=xyz&expires=00000 My Current code: auth_url = 'https://stackexchange.com/oauth/dialog' def authenticate(): scope = "write_access,private_info,read_inbox" url =

Why I get urllib2.HTTPError with urllib2 and no errors with urllib?

由 陌路散爱 提交于 2019-12-03 14:54:43
I have the following simple code: import urllib2 import sys sys.path.append('../BeautifulSoup/BeautifulSoup-3.1.0.1') from BeautifulSoup import * page='http://en.wikipedia.org/wiki/Main_Page' c=urllib2.urlopen(page) This code generates the following error messages: c=urllib2.urlopen(page) File "/usr/lib64/python2.4/urllib2.py", line 130, in urlopen return _opener.open(url, data) File "/usr/lib64/python2.4/urllib2.py", line 364, in open response = meth(req, response) File "/usr/lib64/python2.4/urllib2.py", line 471, in http_response response = self.parent.error( File "/usr/lib64/python2.4

Python调用RESTful API时踩到的各种坑

由 烈酒焚心 提交于 2019-12-03 13:17:06
开篇首先推荐一款REST接口调试的利器，Chrome的扩展程序Advanced REST client，功能十分齐全！使用它来模拟访问接口可以获取到详尽的信息，借助它排查接口是否可用。 问题1：curl访问API成功，但是使用python的urllib2模块访问却返回400。 接口规则如下： POST /v2.0/tokens HTTP/1.0 Host: 192.168.85.183:35357 Content-Type: application/json Accept: application/ json { "auth":{ "passwordCredentials":{ "username":"alan", "password":"admin" }, "tenantName":"swifttenant1" } } cURL模拟访问实现如下： curl -s -d '{"auth":{"passwordCredentials":{"username":"alan","password":"admin"},"tenantName":"swifttenant1"}}' -H "Content-type: application/json" http://192.168.85.183:35357/v2.0/tokens | python -mjson.tool 后台日志： 2013

urllib.urlopen works but urllib2.urlopen doesn't

由 此生再无相见时 提交于 2019-12-03 12:31:56
I have a simple website I'm testing. It's running on localhost and I can access it in my web browser. The index page is simply the word "running". urllib.urlopen will successfully read the page but urllib2.urlopen will not. Here's a script which demonstrates the problem (this is the actual script and not a simplification of a different test script): import urllib, urllib2 print urllib.urlopen("http://127.0.0.1").read() # prints "running" print urllib2.urlopen("http://127.0.0.1").read() # throws an exception Here's the stack trace: Traceback (most recent call last): File "urltest.py", line 5,

Using a session cookie from selenium in urllib2

由 隐身守侯 提交于 2019-12-03 11:58:53
问题 I'm trying to use Selenium to log into a website and then use urllib2 to make RESTy requests. In order for it to work though, I need urllib2 to be able to use the same session Selenium used. The logging in with selenium worked great and I can call self.driver.get_cookies() and I have a list of all the cookies selenium knows about, and it ends up looking a little something like this: [{u'domain': u'my.awesome.web.app.local', u'expiry': 1319230106, u'name': u'ci_session', u'path': u'/', u

ImportError: No module named 'urllib2' Python 3 [duplicate]

由 ♀尐吖头ヾ 提交于 2019-12-03 09:57:37
This question already has answers here : Import error: No module name urllib2 (8 answers) The below code is working fine on Python 2 but on Python 3 I get the error: "ImportError: No module named 'urllib2'" import urllib2 peticion = 'I'm XML' url_test = 'I'm URL' req = urllib2.Request(url=url_test, data=peticion, headers={'Content-Type': 'application/xml'}) respuesta = urllib2.urlopen(req) print(respuesta) print(respuesta.read()) respuesta.open() Please suggest me the reason of error. Thank you. Prashant Puri check StackOverflow Link import urllib.request url = "http://www.google.com/" request

Multithreading for faster downloading

由 与世无争的帅哥 提交于 2019-12-03 09:09:54
How can I download multiple links simultaneously? My script below works but only downloads one at a time and it is extremely slow. I can't figure out how to incorporate multithreading in my script. The Python script: from BeautifulSoup import BeautifulSoup import lxml.html as html import urlparse import os, sys import urllib2 import re print ("downloading and parsing Bibles...") root = html.parse(open('links.html')) for link in root.findall('//a'): url = link.get('href') name = urlparse.urlparse(url).path.split('/')[-1] dirname = urlparse.urlparse(url).path.split('.')[-1] f = urllib2.urlopen

Urllib2 runs fine if i run the program independently but throws error when i add it to a cronjob

由 匿名 (未验证) 提交于 2019-12-03 09:06:55
可以将文章内容翻译成中文,广告屏蔽插件可能会导致该功能失效(如失效，请关闭广告屏蔽插件后再试): 问题: url = "www.someurl.com" request = urllib2.Request(url,header={"User-agent" : "Mozilla/5.0"}) contentString = urllib2.url(request).read() contentFile = StringIO.StringIO(contentString) for i in range(0,2): html = contentFile.readline() print html The above code runs fine from commandline but if i add it to a cron job it throws the following error: File "/usr/lib64/python2.6/urllib2.py", line 409, in _open '_open', req) File "/usr/lib64/python2.6/urllib2.py", line 369, in _call_chain result = func(*args) File "/usr/lib64/python2.6/urllib2.py",

Python 2.6 urlib2 timeout issue

由 匿名 (未验证) 提交于 2019-12-03 08:54:24
可以将文章内容翻译成中文,广告屏蔽插件可能会导致该功能失效(如失效，请关闭广告屏蔽插件后再试): 问题: It seems I cannot get the urllib2 timeout to be taken into account. I did read - I suppose - all posts related to this topic and it seems I'm not doing anything wrong. Am I correct? Many thanks for your kind help. Scenario: I need to check for Internet connectivity before continuing with the remaining of a script. I then wrote a function (Net_Access), which is provided below. When I execute this code with my LAN or Wifi interface connected, and by checking an existing hostname: all is fine as there is no error or problem, thus no timeout. If
订阅 urllib2