问题
I want to disable JavaScript while scraping using scrapy and selenium. Moto of doing that is to increase scraping speed. I found the preference for Firefox driver but not PhantomJS.
firefox_profile = webdriver.FirefoxProfile()
firefox_profile.set_preference("javascript.enabled", False)
driver = webdriver.Firefox(firefox_profile=firefox_profile)
driver.get('http://www.quora.com/')
How can this be done for PhantomJS webdriver?
回答1:
The WebDriver protocol in PhantomJS is a pure JavaScript implementation that is known as Ghostdriver. It makes heavy use of page.evaluate() to access the DOM and there is really no other way to access the DOM, interact with the page or do anything meaningful with PhantomJS. You shouldn't do this.
If you still want to go through with it, this should work:
cap = webdriver.DesiredCapabilities.PHANTOMJS
cap["phantomjs.page.settings.javascriptEnabled"] = False
driver = webdriver.PhantomJS(desired_capabilities=cap)
回答2:
If the site does not require JavaScript, just use scrapy alone. There is no need for selenium. Scrapy is extremely fast for non JavaScript pages.
来源：https://stackoverflow.com/questions/32115673/how-to-disable-javascript-in-phantomjs-through-selenium-webdriver