How to set socks5 proxy in dryscrape python?

问题

I want to scrap some website through tor.

Tor works uses socks proxy. I use dryscrape(library in python) as it also executes javascripts as well.

But I dryscrape can have only http proxy.

Is there some work around to set socks5 proxy for dryscrape or any other way out?

Thanks.

回答1:

I managed to do it using polipo. It can be installed on ubuntu from here.

Polipo can speak socks protocol as well as http protocol. Polipo can be configured to receive requests through http proxy and forward the same requests through socks5 proxy.

I used it as a front-end proxy server to dryscrape, for the back-end tor proxy server.

So, now I set dryscrape's http proxy 127.0.0.1(machine ip) and port no. 8118(default port no. for polipo). Now, Polipo forwards the same request to tor through socks5 proxy.

So, now I can use dryscrape to make request through socks5 proxy.

It is explained in more detail over here. ;)

来源：https://stackoverflow.com/questions/32425180/how-to-set-socks5-proxy-in-dryscrape-python