tor

Scrapy with Privoxy and Tor: how to renew IP

情到浓时终转凉″ 提交于 2019-11-28 04:45:47
I am dealing with Scrapy, Privoxy and Tor. I have all installed and properly working. But Tor connects with the same IP everytime, so I can easily be banned. Is it possible to tell Tor to reconnect each X seconds or connections? Thanks! EDIT about the configuration: For the user agent pool i did this: http://tangww.com/2013/06/UsingRandomAgent/ (I had to put a _ init _.py file as it is said in the comments), and for the Privoxy and Tor I followed http://www.andrewwatters.com/privoxy/ (I had to create the private user and private group manually with the terminal). It worked :) My spider is this

Python - Firefox Headless

a 夏天 提交于 2019-11-27 19:49:47
I've spent the last few days messing around with Selenium, Tor, and Firefox as a combination for multiple tasks. I've managed to write a simple script in Python that takes control of Firefox through Selenium, while Firefox is connected to Tor for privacy. Now, I'm looking for a way to save resources, so I thought of running Firefox in headless mode, which I thought was a common feature but it doesn't seem to be that. I'm looking for a method to do just that. The reason for it being Firefox and not some terminal based browser is because of the extension "TorButton" that I'm using within Firefox

How to route urllib requests through the TOR network? [duplicate]

心已入冬 提交于 2019-11-27 17:45:27
This question already has an answer here: How to make urllib2 requests through Tor in Python? 12 answers How to route urllib requests through the TOR network? This works for me (using urllib2, haven't tried urllib): def req(url): proxy_support = urllib2.ProxyHandler({"http" : "127.0.0.1:8118"}) opener = urllib2.build_opener(proxy_support) opener.addheaders = [('User-agent', 'Mozilla/5.0')] return opener.open(url).read() print req('http://google.com') Tor works as a proxy, right? So ask yourself "How do I use proxies in urllib?" Now, when I look at the docs, first thing I see is urllib.urlopen

How to change the Tor exit node programmatically to get a new IP?

天大地大妈咪最大 提交于 2019-11-27 17:20:20
I have Tor running on my computer, and I need to change the Tor exit node every five minutes. For example, if I start using Tor via some exit node, then in 5 minutes I want Tor to change to an exit node with a different IP address. How can I do this? Tor, as far as I know, is listening to port 8051 on localhost. What commands can I send to this port to make Tor build a new chain, so that I can get another IP address? (echo authenticate '""'; echo signal newnym; echo quit) | nc localhost 9051 Method 1: HUP sudo killall -HUP tor Then check that your IP has changed with: curl --socks5 127.0.0.1

using tor as a SOCKS5 proxy with python urllib2 or mechanize

*爱你&永不变心* 提交于 2019-11-27 16:53:05
问题 My goal is to use python's mechanize with a tor SOCKS proxy. I am not using a GUI with the following Ubuntu version: Description: Ubuntu 12.04.1 LTS Release: 12.04 Codename: precise Tor is installed and is listening on port 9050 according to the nmap scan: Starting Nmap 5.21 ( http://nmap.org ) at 2013-01-22 00:50 UTC Nmap scan report for localhost (127.0.0.1) Host is up (0.000011s latency). Not shown: 996 closed ports PORT STATE SERVICE 22/tcp open ssh 80/tcp open http 3306/tcp open mysql

Using Selenium WebDriver with Tor

独自空忆成欢 提交于 2019-11-27 12:07:08
Because Tor Browser Bundle is just a patched version of Firefox, it seems that it should be possible to use a FirefoxDriver with Tor Browser. This is what I've tried so far: String torPath = "C:\\Users\\My User\\Desktop\\Tor Browser\\Start Tor Browser.exe"; String profilePath = "C:\\Users\\My User\\Desktop\\Tor Browser\\Data\\Browser\\profile.default"; FirefoxProfile profile = new FirefoxProfile(new File(profilePath)); FirefoxBinary binary = new FirefoxBinary(new File(torPath)); FirefoxDriver driver = new FirefoxDriver(binary, profile); driver.get("http://www.google.com"); This results in a

Open tor browser with selenium

末鹿安然 提交于 2019-11-27 11:29:17
Is it possible to make selenium use the TOR browser? Does anyone have any code they could copy-paste? Don't use the TBB, just set the correct proxy settings in whatever browser you're using. In FF for example, like this: #set some privacy settings ff_prof.set_preference( "places.history.enabled", False ) ff_prof.set_preference( "privacy.clearOnShutdown.offlineApps", True ) ff_prof.set_preference( "privacy.clearOnShutdown.passwords", True ) ff_prof.set_preference( "privacy.clearOnShutdown.siteSettings", True ) ff_prof.set_preference( "privacy.sanitize.sanitizeOnShutdown", True ) ff_prof.set

How to contribute on github anonymously via Tor?

試著忘記壹切 提交于 2019-11-27 09:59:42
问题 I would like to contribute anonymously to projects on github. Not to cause mischief, more in the spirit of anonymous donations. The tool of choice for being anonymous online seems to be TOR, which works well for almost anything you can do in a browser. However, to contribute on github, it appears necessary to use the command line interface, or the Mac app. How can I channel my git operations in this setup through Tor? And how can I verify that this is actually what is happening? Edit: please

How can I connect to a Tor hidden service using cURL in PHP?

大城市里の小女人 提交于 2019-11-27 09:55:04
I'm trying to connect to a Tor hidden service using the following PHP code: $url = 'http://jhiwjjlqpyawmpjx.onion/' $ch = curl_init(); curl_setopt($ch, CURLOPT_URL, $url); curl_setopt($ch, CURLOPT_RETURNTRANSFER, true); curl_setopt($ch, CURLOPT_PROXY, "http://127.0.0.1:9050/"); curl_setopt($ch, CURLOPT_PROXYTYPE, CURLPROXY_SOCKS5); $output = curl_exec($ch); $curl_error = curl_error($ch); curl_close($ch); print_r($output); print_r($curl_error); When I run it, I get the following error: Couldn't resolve host name However, when I run the following command from my command line in Ubuntu: curl -v -

How to change Tor identity in Python?

倖福魔咒の 提交于 2019-11-27 06:02:22
I have the following script: import socks import socket socks.setdefaultproxy(socks.PROXY_TYPE_SOCKS5, "127.0.0.1", 9050) socket.socket = socks.socksocket import urllib2 print(urllib2.urlopen("http://www.ifconfig.me/ip").read()) which uses tor and SocksiPy Now I want to change tor identity with each request, for example: for i in range(0, 10): #somehow change tor identity print(urllib2.urlopen("http://www.ifconfig.me/ip").read()) How can I do this? Tor wrote a new TOR control library in Python, stem . It can be found on PyPI . They provide some nice tutorials how to work with it, one of them