urllib2

I'm having troubles connecting with https using urllib2 under Python 2.7.10. Any thoughts what I'm missing? Python 2.7.10 (default, Jun 18 2015, 10:53:24) [GCC 4.4.5] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import ssl, urllib2 >>> ssl.HAS_SNI True >>> ssl.OPENSSL_VERSION 'OpenSSL 0.9.8o 01 Jun 2010' >>> opener = urllib2.build_opener() >>> opener.open('https://twitrss.me/') Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/usr/local/python2.7/lib/python2.7/urllib2.py", line 431, in open response = self._open(req, data)

I have this simple minimal 'working' example below that opens a connection to google every two seconds. When I run this script when I have a working internet connection, I get the Success message, and when I then disconnect, I get the Fail message and when I reconnect again I get the Success again. So far, so good. However, when I start the script when the internet is disconnected, I get the Fail messages, and when I connect later, I never get the Success message. I keep getting the error: urlopen error [Errno -2] Name or service not known What is going on? import urllib2, time while True: try

Is there a good way to download a lot of files en masse using python? This code is speedy enough for downloading about 100 or so files. But I need to download 300,000 files. Obviously they are all very small files (or I wouldn't be downloading 300,000 of them :) ) so the real bottleneck seems to be this loop. Does anyone have any thoughts? Maybe using MPI or threading? Do I just have to live with the bottle neck? Or is there a faster way, maybe not even using python? (I included the full beginning of the code just for completeness sake) from __future__ import division import pandas as pd

背景 在运行一个用python2写的知识图谱项目时，在进行到里面获取百度百科数据时，报了编码异常错误 UnicodeEncodeError: 'ascii' codec can't encode characters in position 10-13: ordinal not in range(...) 解决方法 不要用urllib.request或urllib2来请求，换成requests，如下 import requests data = requests.request(url=url, headers=headers, method="GET").content 其中的headers为字典，可定义如下 headers = { "user-agent": "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/49.0.2623.22 Safari/537.36 SE 2.X MetaSr 1.0" } 结语 python3和python2的差别还是很大，比如urllib2直接没了，可以直接用requests，在运行一些python2的项目时要注意，或者python2和python3都装起来也是可以。 来源： https://blog.csdn.net/qq

I am making several http requests to a particular host using python's urllib2 library. Each time a request is made a new tcp and http connection is created which takes a noticeable amount of time. Is there any way to keep the tcp/http connection alive using urllib2? Corey Goldberg If you switch to httplib , you will have finer control over the underlying connection. For example: import httplib conn = httplib.HTTPConnection(url) conn.request('GET', '/foo') r1 = conn.getresponse() r1.read() conn.request('GET', '/bar') r2 = conn.getresponse() r2.read() conn.close() This would send 2 HTTP GETs on

I'm perplexed as to why I'm not able to download the entire contents of some JSON responses from FriendFeed using urllib2 . >>> import urllib2 >>> stream = urllib2.urlopen('http://friendfeed.com/api/room/the-life-scientists/profile?format=json') >>> stream.headers['content-length'] '168928' >>> data = stream.read() >>> len(data) 61058 >>> # We can see here that I did not retrieve the full JSON ... # given that the stream doesn't end with a closing } ... >>> data[-40:] 'ce2-003048343a40","name":"Vincent Racani' How can I retrieve the full response with urllib2? Best way to get all of the data: