urllib

IOError: [Errno socket error] [Errno 11004] getaddrinfo failed

喜夏-厌秋 提交于 2019-11-28 02:05:18
问题 I am beginner python prorammer. With 2.7.2, Windows 7, built-in interpreter, and three libraries. I am trying to do this, with error. I appreciate any help? import os import urllib import socket DISNEY_URL = 'http://www.sec.gov/Archives/edgar/data/1001039/000119312511321340/dis-20111001.xml' #Neither of these seem to work when opening with urllib.urlopen becaue of the error: #I/O error(socket error): [Errno 11004] getaddrinfo failed DISNEY_LOCAL = 'file://C:/Users/Nate/Desktop/Education

python requests is slow

对着背影说爱祢 提交于 2019-11-28 01:30:45
问题 I am developing a download manager. Using the requests module in python to check for a valid link (and hopefully broken links). My code for checking link below: url='http://pyscripter.googlecode.com/files/PyScripter-v2.5.3-Setup.exe' r = requests.get(url,allow_redirects=False) #this line takes 40 seconds if r.status_code==200: print "link valid" else: print "link invalid" Now, the issue is this takes approximately 40 seconds to perform this check, which is huge. My question is how can I speed

gaierror: [Errno -2] Name or service not known

拜拜、爱过 提交于 2019-11-28 00:52:00
问题 def make_req(data, url, method='POST') params = urllib.urlencode(data) headers = {"Content-type": "application/x-www-form-urlencoded", "Accept": "text/plain", } conn = httplib.HTTPSConnection(url) conn.request(method, url, params, headers) response = conn.getresponse() response_data = response.read() conn.close() But it is throwing: in create_connection for res in getaddrinfo(host, port, 0, SOCK_STREAM): gaierror: [Errno -2] Name or service not known What is the reason ? What is this error?

urllib.request.urlretrieve with proxy?

安稳与你 提交于 2019-11-27 22:23:10
somehow I can't download files trough a proxyserver, and I don't know what i have done wrong. I just get a timeout. Any advice? import urllib.request urllib.request.ProxyHandler({"http" : "myproxy:123"}) urllib.request.urlretrieve("http://myfile", "file.file") You need to use your proxy-object, not just instanciate it (you created an object, but didn't assign it to a variable and therefore can't use it). Try using this pattern: #create the object, assign it to a variable proxy = urllib.request.ProxyHandler({'http': '127.0.0.1'}) # construct a new opener using your proxy settings opener =

How to use urllib in python 3?

点点圈 提交于 2019-11-27 22:19:18
Here is my problem with urllib in python 3. I wrote a piece of code which works well in Python 2.7 and is using urllib2. It goes to the page on Internet (which requires authorization) and grabs me the info from that page. The real problem for me is that I can't make my code working in python 3.4 because there is no urllib2, and urllib works differently; even after few hours of googling and reading I got nothing. So if somebody can help me to solve this, I'd really appreciate that help. Here is my code: request = urllib2.Request('http://mysite/admin/index.cgi?index=127') base64string = base64

Javascript access another webpage

血红的双手。 提交于 2019-11-27 21:26:05
问题 I know very, very little of javascript, but I'm interested in writing a script which needs information from another webpage. It there a javascript equivalent of something like urllib2? It doesn't need to be very robust, just enough to process a simple GET request, no need to store cookies or anything and store the results. 回答1: There is the XMLHttpRequest, but that would be limited to the same domain of your web site, because of the Same Origin Policy. However, you may be interested in

Is there a unicode-ready substitute I can use for urllib.quote and urllib.unquote in Python 2.6.5?

寵の児 提交于 2019-11-27 20:22:50
Python's urllib.quote and urllib.unquote do not handle Unicode correctly in Python 2.6.5. This is what happens: In [5]: print urllib.unquote(urllib.quote(u'Cataño')) --------------------------------------------------------------------------- KeyError Traceback (most recent call last) /home/kkinder/<ipython console> in <module>() /usr/lib/python2.6/urllib.pyc in quote(s, safe) 1222 safe_map[c] = (c in safe) and c or ('%%%02X' % i) 1223 _safemaps[cachekey] = safe_map -> 1224 res = map(safe_map.__getitem__, s) 1225 return ''.join(res) 1226 KeyError: u'\xc3' Encoding the value to UTF8 also does

Python standard library to POST multipart/form-data encoded data

大城市里の小女人 提交于 2019-11-27 20:07:57
I would like to POST multipart/form-data encoded data. I have found an external module that does it: http://atlee.ca/software/poster/index.html however I would rather avoid this dependency. Is there a way to do this using the standard libraries? thanks Martin v. Löwis The standard library does not currently support that . There is cookbook recipe that includes a fairly short piece of code that you just may want to copy, though, along with long discussions of alternatives. ticapix It's an old thread but still a popular one, so here is my contribution using only standard modules. The idea is the

Only add to a dict if a condition is met

孤人 提交于 2019-11-27 19:35:13
I am using urllib.urlencode to build web POST parameters, however there are a few values I only want to be added if a value other than None exists for them. apple = 'green' orange = 'orange' params = urllib.urlencode({ 'apple': apple, 'orange': orange }) That works fine, however if I make the orange variable optional, how can I prevent it from being added to the parameters? Something like this (pseudocode): apple = 'green' orange = None params = urllib.urlencode({ 'apple': apple, if orange: 'orange': orange }) I hope this was clear enough, does anyone know how to solve this? You'll have to add

How to download a file over http with authorization in python 3.0, working around bugs?

淺唱寂寞╮ 提交于 2019-11-27 18:25:48
问题 I have a script that I'd like to continue using, but it looks like I either have to find some workaround for a bug in Python 3, or downgrade back to 2.6, and thus having to downgrade other scripts as well... Hopefully someone here have already managed to find a workaround. The problem is that due to the new changes in Python 3.0 regarding bytes and strings, not all the library code is apparently tested. I have a script that downloades a page from a web server. This script passed a username