Tor doesn't work with urllib2

Question

I am trying to use tor for anonymous access through privoxy as a proxy using urllib2.

System info: Ubuntu 14.04, recently upgraded from 13.10 through dist-upgrade.

This is a piece of code I am using for test purposes:

import urllib2


def req(url):
    proxy_support = urllib2.ProxyHandler({"http": "127.0.0.1:8118"})
    opener = urllib2.build_opener(proxy_support)
    opener.addheaders = [('User-agent', 'Mozilla/5.0')]
    return opener.open(url).read()

print req('https://check.torproject.org')

The above outputs a page with a sorry, but you don't use Tor message.

As for my configurations:

/etc/tor/torrc

ControlPort 9051
## If you enable the controlport, be sure to enable one of these
## authentication methods, to prevent attackers from accessing it.
#HashedControlPassword 16:872860B76453A77D60CA2BB8C1A7042072093276A3D701AD684053EC4C
HashedControlPassword 16:B3B0BA631D41D6FE601488FC7A9A2E80AB7815580C1C78F2865F30F63B

/etc/privoxy/config

forward-socks5 / localhost:9050 .

---

$ sudo netstat -ntap | grep tor outputs:

tcp        0      0 127.0.0.1:9050          0.0.0.0:*               LISTEN      1045/tor        
tcp        0      0 127.0.0.1:9051          0.0.0.0:*               LISTEN      1045/tor        
tcp        0      0 10.0.0.94:56736         85.17.190.83:9002       ESTABLISHED 1045/tor        
tcp        0      0 10.0.0.94:60558         50.7.110.118:9001       ESTABLISHED 1045/tor        
tcp        0      0 10.0.0.94:43206         62.210.236.135:443      ESTABLISHED 1045/tor   

$ sudo netstat -ntap | grep privoxy outputs:

tcp        0      0 127.0.0.1:8118          0.0.0.0:*               LISTEN      887/privoxy 

And yet, all requests are received at the server with my actual

Edit:

Below code, which is using the requests library, gives the same output:

import requests


def req(url):
    proxies = {"http": "http://127.0.0.1:8118"}
    return requests.get(url, proxies=proxies).text

print req('https://check.torproject.org')

Tor log doesn't indicate any problem:

Apr 22 15:44:34.000 [notice] Bootstrapped 100%: Done.
Apr 22 15:45:03.000 [notice] Catching signal TERM, exiting cleanly.
Apr 22 16:00:57.000 [notice] Tor 0.2.4.20 (git-0d50b03673670de6) opening log file.
Apr 22 16:00:57.000 [notice] Parsing GEOIP IPv4 file /usr/share/tor/geoip.
Apr 22 16:00:57.000 [notice] Parsing GEOIP IPv6 file /usr/share/tor/geoip6.
Apr 22 16:00:57.000 [warn] OpenSSL version from headers does not match the version we're running with. If you get weird crashes, that might be why. (Compiled with 1000105f: OpenSSL 1.0.1e 11 Feb 2013; running wi$
Apr 22 16:00:58.000 [notice] We now have enough directory information to build circuits.
Apr 22 16:00:58.000 [notice] Bootstrapped 80%: Connecting to the Tor network.
Apr 22 16:00:59.000 [notice] Bootstrapped 85%: Finishing handshake with first hop.
Apr 22 16:00:59.000 [notice] Bootstrapped 90%: Establishing a Tor circuit.
Apr 22 16:01:00.000 [notice] Tor has successfully opened a circuit. Looks like client functionality is working.
Apr 22 16:01:00.000 [notice] Bootstrapped 100%: Done.

Answer 1

Maybe this answer is related.

In a nutshell, you're using 127.0.0.1:8118 as a proxy url, and http as your protocol, but the protocol should be https instead.