headless-browser | 易学教程

HtmlUnit not creating HtmlPage object

阅读更多关于 HtmlUnit not creating HtmlPage object

I'm very new to HtmlUnit and I'm trying to scrape a website that uses Javascript to edit the code. I heard HtmlUnit was the best way to go as it returns the final code using a headless browser. However as you will see I cannot even get past creating a HtmlPage object without getting a huge and impossible to understand exception thrown (at least given my virtually null experience with HtmlUnit). Here is my code: import com.gargoylesoftware.htmlunit.*; import com.gargoylesoftware.htmlunit.html.HtmlPage; public class Main { public static void main(String[] args) { Main scraper = new Main();

How do I use Selenium Webdriver on headless Chrome?

阅读更多关于 How do I use Selenium Webdriver on headless Chrome?

I'm learning to use Selenium for basic things like taking a screenshot, scraping, and testing and would like to use it with headless Chrome, which is now stable as of Chrome 59. I have been able to take a screenshot using the 'selenium-webdriver' gem and chromedriver, but not headless. Here is the ruby script that I am running which hangs after starting to initialize the driver require 'rubygems' require 'selenium-webdriver' Selenium::WebDriver.logger.level = :debug p 'initializing driver' driver = Selenium::WebDriver.for :chrome, switches: %w[--headless --disable-gpu --screenshot --hide

Want to scrape table using puppeteer.js. How can I get all rows, iterate through rows and then get “td's” for each row

阅读更多关于 Want to scrape table using puppeteer.js. How can I get all rows, iterate through rows and then get “td's” for each row

I have puppeteer js setup and was able get all rows using let rows = await page.$$eval('#myTable tr', row => row); Now I want for each row to get "td's" and then get inner text from those. Basically I want to do this: var tds = myRow.querySelectorAll("td"); where myRow is a table row, with puppeteer.js One way to achieve this is to use evaluate that first gets an array of all the TD's then returns the textContent of each TD const puppeteer = require('puppeteer'); const html = ` <html> <body> <table> <tr><td>One</td><td>Two</td></tr> <tr><td>Three</td><td>Four</td></tr> </table> </body> </html>

Injecting javascript into zombie.js

阅读更多关于 Injecting javascript into zombie.js

问题 Hi I was wondering if there is the ability in node js and zombie js to inject javascript files in to the headless browser, similar to what you can do with phantomjs. For example in phantom js you would do: page.injectJs("amino/TVI.js") I have used phantomjs and it does do what I want it to do but however I am testing other options due to the high memory required by using phantom js. 回答1: you can append script tag into document object since it support DOM API in zombie. The following example

Execute browser page/javascript from a script/command-line

阅读更多关于 Execute browser page/javascript from a script/command-line

Hope this isnt a stupid question. I have recently had an idea about something which I am very curious about. I am a fan of Node.js (not really relevent here I think) and the V8 engine but I was wondering if its possible to run a browser (get it to execute JS) but INTERNALLY. What I mean by that is to create a program (possibly using the V8 engine) which can open a page (as if in the browser) and execute its javascript. For instance say I have the below file hosted on www.mysite.co.uk/home.php <!DOCTYPE html> <html> <head> <script> function myFunction() { //javascript AJAX call to www.mysite.co

WebDriverException: Element is not clickable at point

阅读更多关于 WebDriverException: Element is not clickable at point

问题 Complete error org.openqa.selenium.WebDriverException: Element is not clickable at point (931, 23). Other element would receive the click: Running selenium tests using Jenkins in headless browser In following environment OS- CentOS 6 and Xvfb installed in same machine Xvfb screen resolution - Xvfb :1 -screen 0 1280x1024x16 & Driver - Firefox driver Selenium webdriver version - 2.52.0 The same test case working fine in windows os with firefox 回答1: Try this. Install Xvfb plugin on Jenkins.

How to run browsers(chrome, IE and firefox) in headless mode?

阅读更多关于 How to run browsers(chrome, IE and firefox) in headless mode?

I want to integrate JSTestDriver with jenkins-CI. problem is that the build server does not have display. is there a way to run browsers headlessly? Thanks If your server is Linux, as mentioned, you can't run IE. But you can run other browsers like Firefox. The trick is to install XVFB on your Jenkins server which allows Firefox to run headless. I've used this setup and it works well. http://www.alittlemadness.com/2008/03/05/running-selenium-headless/ Google Chrome now ( Chrome 59 for Mac and Linux, Chrome 60 for Windows ) has a headless mode built-in. It works cross-platform, and doesn't

Using QWebEngine to render an image

阅读更多关于 Using QWebEngine to render an image

I'm looking to replace QWebKit with QWebEngine in my headless renderer. I initialise the page with load() and connect a slot to loadFinished() to generate the final .PNG image. This used to work fine with WebKit but fails with QWebEngine . Code is as follows... _webView = new QWebEngineView(); .... // Render the HTML to an image QPainter painter(&image); _webView->page()->view()->render(&painter); painter.end(); I receive the following errors : "Asking for share context for widget that does not have a window handle" "QOpenGLWidget: Cannot be used without a context shared with the toplevel".

How to delete existing text from input using Puppeteer?

阅读更多关于 How to delete existing text from input using Puppeteer?

I'm trying to test amending text in an editable input which contains the title of the current record - and I want to able to test editing such text, replacing it with something else. I know I can use await page.type('#inputID', 'blah'); to insert "blah" into the textbox (which in my case, having existing text, only appends "blah"), however, I cannot find any page methods 1 that allow deleting or replacing existing text. You can use page.evaluate to manipulate DOM as you see fit: await page.evaluate( () => document.getElementById("inputID").value = "") However sometimes just manipulating a

Selenium Pyvirtualdisplay Hangs on starting

阅读更多关于 Selenium Pyvirtualdisplay Hangs on starting

I have a selenium test which I am trying to run Headlessly on my ubuntu server. It uses pyvirtualdisplay and xephyr. Though the script is running fine in my local system, it hangs when I run it in the server. I have tried to clear memory and restarting the server. But no luck. The script looks something like this : from selenium import webdriver from pyvirtualdisplay import Display display = Display(visible=0, size=(1366, 768)) display.start() driver = webdriver.Firefox() driver.set_window_size(1366, 768) driver.get ("http://www.google.com/") Any help? Okay, finally I got through it. sudo apt