Phantom.js / Casper.js with rotating proxy?

断了今生、忘了曾经 提交于 2020-01-01 09:25:10

问题


I have a simple goal: load webpages with either phantom.js (out of the box) or casper.js (nice and easier) but using proxy and rotate it from a list if current one is bad (i.e. webpage loads fail or something like that).

I know casper.js has --proxy param but it dictates the user to specify only ONE proxy and use it during runtime.

Question #1 is: how to rotate proxy on the fly programmatically?

I did some research and found this node-requester but it's not integrated with casper.js. I tried to extract out just the proxy feature in the code but didn't really understand how it works in the nutshell (I am not that smart I guess).

So question #2: is there some simple implementation of proxy rotation that works with either phantom.js or casper.js?

I prefer to use the fancy casper.js though but will go down with phantom.js bare as well.


回答1:


I had the same issue a while back, I worked with PhantomJS. The solution we ended up with was running PhantomJS as a child process of a larger Java/Scala server which then handled failures and assigned the different proxies when needed (by rerunning with different params in the --proxy arg).



来源:https://stackoverflow.com/questions/16746603/phantom-js-casper-js-with-rotating-proxy

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!