headless-browser

Using a headless browser with Heroku Rails Unicorn stack

我们两清 提交于 2019-11-30 15:53:40
I have seen posts that you can change the build on Heroku, to use PhantomJS with the Cedar stack, but is it possible to run a headless browser on the unicorn stack? Yes, and someone already wrote a buildpack for including PhantomJS (the internet and open source are amazing). If you want to run headless PhantomJS on a Heroku dyno, you will need to: Switch to Heroku multi-buildpack: https://github.com/ddollar/heroku-buildpack-multi Setup Ruby buildpack for your app: https://github.com/heroku/heroku-buildpack-ruby Setup PhantomJS buildpack for your needs: https://github.com/stomita/heroku

Output client-side console with casper/phantomjs

我与影子孤独终老i 提交于 2019-11-30 11:04:49
Going through the casperjs documentation I couldn't find where I could see the console.log from client-side javascript. Is this possible? I'm not really sure to fully understand your question, but you can do something like the following: var casper = require('casper').create({ logLevel: "debug" }); casper.on('remote.message', function(message) { this.echo(message); }); casper.start('http://google.com/', function() { this.evaluate(function sendLog(log) { // you can access the log from page DOM console.log('from the browser, I can tell you there are ' + log.length + ' entries in the log'); },

c# headless browser with javascript support for crawler

独自空忆成欢 提交于 2019-11-30 08:36:59
问题 Could anyone suggest headless browser for .NET that supports cookies and authomatically javascript execution? 回答1: Selenium+HtmlUnitDriver/GhostDriver is exactly what you are looking for. Oversimplified, Selenium is library for using variety of browsers for automation purposes - testing, scraping, task automation. There are different WebDriver classes with which you can operate an actual browser. HtmlUnitDriver is a headless one. GhostDriver is a WebDriver for PhantomJS, so you can write C#

Can you deploy Watir on Heroku to generate HTML Snapshots? If so, how?

心不动则不痛 提交于 2019-11-30 07:46:51
I would like to generate HTML Snapshots using Watir , hosted on Heroku . Google's Full Specification for Making AJAX Applications Crawlable suggests using HTMLUnit ... see How do I create an HTML snapshot? point #3. HtmlUnit is a Java-only headless browser emulator; and unfortunately jRuby is not an option on Heroku. So HtmlUnit is ruled out (to my knowledge). If you're interested I have another question open regarding HtmlUnit as a service hosted on Google App Engine... Making AJAX Applications Crawlable? How to build a simple web service on Google App Engine to produce HTML Snapshots? ...

Problem in HtmlUnit API for Java (Headless Browser)?

 ̄綄美尐妖づ 提交于 2019-11-30 07:41:16
问题 I am using HtmlUnit headless browser to browse this webpage (you can see the webpage to have a better understanding of the problem). I have set the select's value to "1" by the following commands final WebClient webClient = new WebClient(BrowserVersion.INTERNET_EXPLORER_7); try { // Configuring the webClient webClient.setJavaScriptEnabled(true); webClient.setThrowExceptionOnScriptError(false); webClient.setCssEnabled(true); webClient.setUseInsecureSSL(true); webClient.setRedirectEnabled(true)

Using CefSharp.Offscreen to retrieve a web page that requires Javascript to render

主宰稳场 提交于 2019-11-30 07:35:47
I have what is hopefully a simple task, but it's going to take someone that's versed in CefSharp to solve it. I have an url that I want to retrieve the HTML from. The problem is this particular url doesn't actually distribute the page on a GET. Instead, it pushes a mound of Javascript to the browser, which then executes and produces the actual rendered page. This means that the usual approaches involving HttpWebRequest and HttpWebResponse aren't going to work. I've looked at a number of different "headless" options, and the one that I think best meets my needs for a number of reasons is

Headless browser testing with download functionality?

余生颓废 提交于 2019-11-30 07:30:30
问题 I've been looking for a solution to do headless testing in osx. But I need the ability to save files returned by the server. I've tested selenium, phantomjs, casperjs and have looked into anything I could find online. none of them supports downloading. am I missing something? are there any headless browser/testing frameworks that support downloads? 回答1: What you can do is: start a virtual display (see Xvfb) start up Firefox browser with preferences configured to automatically save csv files

What is a good headless browser to run with protractor?

China☆狼群 提交于 2019-11-30 06:41:54
New User here. After hours of building my smoke and regression tests, I found out after reading many cases online that phantomjs is known to be a trouble to run with protractor. Jenkins has been running phantomjs for all the tasks it has been given so far. They need these tests to run as part of ci which does not have a windowing system installed. So I would appreciate it if there is a recommendation for completely headless browser or a headless chrome(that would be most beneficial) and a step by step to set it up. I already have a conf.js and a e2e.conf.js file. My code works perfectly fine

How do I use Headless Chrome in Chrome 60 on Windows 10?

筅森魡賤 提交于 2019-11-30 06:04:48
I've been looking at the following article about Headless Chrome: https://developers.google.com/web/updates/2017/04/headless-chrome I just upgraded Chrome on Windows 10 to version 60, but when I run either of the following commands from the command line, nothing seems to happen: chrome --headless --disable-gpu --dump-dom https://www.google.com/ chrome --headless --disable-gpu --print-to-pdf https://www.google.com/ And I'm running all of these commands from the following path (the default installation path for Chrome on Windows): C:\Program Files (x86)\Google\Chrome\Application\ When I run the

How to set the browser window size when using `google-chrome --headless`?

那年仲夏 提交于 2019-11-29 23:45:07
Tried setting the browser size on chrome --headless by using Selenium webdriver commands. I get this webdriver error: - Failed: unknown error: cannot get automation extension from unknown error: page could not be found: chrome-extension://aapnijgdinlhnhlmodcfapnahmbfebeb/_generated_background_page.html (Session info: headless chrome=58.0.3029.81) (Driver info: chromedriver=2.29.461571 (8a88bbe0775e2a23afda0ceaf2ef7ee74e822cc5),platform=Linux 4.4.0-72-generic x86_64) (WARNING: The server did not provide any stacktrace information) Command duration or timeout: 10.07 seconds Build info: version: