headless-browser

I'm trying to speed up Selenium/PhantomJS webscraper in Python by preventing download of CSS/other resources. All I need to download is img src and alt tags. I've found this code: page.onResourceRequested = function(requestData, request) { if ((/http:\/\/.+?\.css/gi).test(requestData['url']) || requestData['Content-Type'] == 'text/css') { console.log('The url of the request is matching. Aborting: ' + requestData['url']); request.abort(); } }; via: How can I control PhantomJS to skip download some kind of resource? How/where can I implement this code in Selenium driven by Python? Or, is there

Actually I want to run my selenium code on server where dont want open the any browser. But I am confused which webdriver is use for the server which do all task (where I download some files from some site and store in my server). To execute your Test Suite through Selenium without opening any browser you can use any of the Browser Client from the following list : Headless Chrome : Here you can find an working example . Headless Firefox : Here you can find an working example . PhantomJS : Here you can find an working example . HtmlUnitDriver : Here you can find an working example . headless

I have been using HtmlUnit (the developers did a great job) as an headless browser for some of my previous applications but the javascript support isn't working for some website that my next application will be accessing. I heard about QtWebKit binding for Python but my application will be in Java or is there a Java binding for WebKit or QtWebKit? Does anyone know a good headless browser for Java with full javascript support? Haroldo_OK Nathan Ridley's answer to another similar question is the most complete one I've found so far. Anyway, if everything fails, you could use a Python or JS

I'm looking to automate some web interactions, namely periodic download of files from a secure website. This basically involves entering my username/password and navigating to the appropriate URL. I tried simple scripting in Python, followed by more sophisticated scripting, only to discover this particular website is using some obnoxious javascript and flash based mechanism for login, rendering my methods useless. I then tried HTMLUnit, but that doesn't seem to want to work either. I suspect use of Flash is the issue. I don't really want to think about it any more, so I'm leaning towards

I am downloading HTML pages that have data defined in them in the following way: ... <script type= "text/javascript"> window.blog.data = {"activity":{"type":"read"}}; </script> ... I would like to extract the JSON object defined in 'window.blog.data'. Is there a simpler way than parsing it manually? (I am looking into Beautiful Soap but can't seem to find a method that will return the exact object without parsing) Thanks Edit: Would it be possible and more correct to do this with a python headless browser (e.g., Ghost.py)? jfs BeautifulSoup is an html parser; you also need a javascript parser

I'd like to use Selenium to automate a few web tasks (not for testing). I think I have Selenium RC Server installed, but have no way of writing "test scripts" since I can't find a client driver in PHP (see: http://seleniumhq.org/download/ ). Is there a way for me to use Selenium with PHP? This seems to suggest I need PHPUnit http://www.phpunit.de/manual/current/en/selenium.html . I just want to automate a few tasks, not get involved with a full suite of testing. Try Following things Get Phpunit installed and working Also have JAVA sdk & jre on your pc. Now record test cases using selenium IDE.

Is there a headless browser library for PHP? Would like something that has a JS engine built into it. FOSS preferred. Anthony Hatzopoulos PhantomJS - http://phantomjs.org/ PhantomJS is a headless WebKit with JavaScript API. It has fast and native support for various web standards: DOM handling, CSS selector, JSON, Canvas, and SVG. You can couple it with something like php-PhantomjsRunner (now deprecated) if you want or bake your own . When setup and ready to start testing with PhantomJS, pick out one of the various Testing Frameworks at your disposal. I recently also came across this tutorial