headless-browser

How to get all links from the DOM?

由 时光怂恿深爱的人放手 提交于 2020-12-29 06:51:22
问题 According to https://github.com/GoogleChrome/puppeteer/issues/628, I should be able to get all links from < a href="xyz" > with this single line: const hrefs = await page.$$eval('a', a => a.href); But when I try a simple: console.log(hrefs) I only get: http://example.de/index.html ... as output which means that it could only find 1 link? But the page definitely has 12 links in the source code / DOM. Why does it fail to find them all? Minimal example: 'use strict'; const puppeteer = require(

How to apply javascript to html simulating a browser

由 丶灬走出姿态 提交于 2020-07-21 06:42:28
问题 I've already searched on the Internet how to "create" a simple headless browser, because I was interested to know how does a Browser works internally. I'd like to implement a simple headless-browser. What I mean is: suppose you have an html string, and a javascript string, both as a result of a HttpRequest to the server; how can I apply the javascript into the html string? For example: I requested to an X server the html source file, and I obtained in the response this: <html> <head> <script

Unable to invoke firefox headless

由 那年仲夏 提交于 2020-06-25 05:19:29
问题 I would like to know why i'm getting the following error message and how can i fix it: Traceback (most recent call last): File "teste.py", line 30, in <\module> main() File "teste.py", line 24, in main driver = connectFirefox(defineOptions()) File "teste.py", line 18, in connectFirefox driver = webdriver.Firefox(firefox_options=options) File "/usr/lib/python3.6/site-packages/selenium/webdriver/firefox /webdriver.py", line 125, in init if options.binary is not None: AttributeError: 'Options'

How to avoid being detected as bot on Puppeteer and Phantomjs?

由 Deadly 提交于 2020-05-24 11:32:53
问题 Puppeteer and PhantomJS are similar. The issue I'm having is happening for both, and the code is also similar. I'd like to catch some informations from a website, which needs authentication for viewing those informations. I can't even access home page because it's detected like a "suspicious activity", like the SS: https://i.imgur.com/p69OIjO.png I discovered that the problem doesn't happen when I tested on Postman using a header named Cookie and the value of it's cookie caught on browser,

How to avoid being detected as bot on Puppeteer and Phantomjs?

由 谁都会走 提交于 2020-05-24 11:32:10
问题 Puppeteer and PhantomJS are similar. The issue I'm having is happening for both, and the code is also similar. I'd like to catch some informations from a website, which needs authentication for viewing those informations. I can't even access home page because it's detected like a "suspicious activity", like the SS: https://i.imgur.com/p69OIjO.png I discovered that the problem doesn't happen when I tested on Postman using a header named Cookie and the value of it's cookie caught on browser,

Headless automation with Nodejs Selenium Webdriver

由 烂漫一生 提交于 2020-05-09 18:06:12
问题 I am working with an automation tool which has to be deployed inside an ubuntu server, my wonder is if is possible to use chrome in a silent way with Selenium Webdriver. I've tried the following code so far, but it keeps opening the browser (I'm doing the tests in a Windows 10): var webdriver = require('selenium-webdriver'), chrome = require('selenium-webdriver/chrome') By = webdriver.By, until = webdriver.until, options = new chrome.Options(); options.addArguments('--headless'); var path =

How can I dynamically inject functions to evaluate using Puppeteer?

由 [亡魂溺海] 提交于 2020-04-27 18:44:35
问题 I am using Puppeteer for headless Chrome. I wish to evaluate a function inside the page that uses parts of other functions, defined dynamically elsewhere. The code below is a minimal example / proof. In reality functionToInject() and otherFunctionToInject() are more complex and require the pages DOM. const browser = await puppeteer.launch(); const page = await browser.newPage(); await page.goto(someURL); var functionToInject = function(){ return 1+1; } var otherFunctionToInject = function

How can I dynamically inject functions to evaluate using Puppeteer?

由 二次信任 提交于 2020-04-27 18:29:33
问题 I am using Puppeteer for headless Chrome. I wish to evaluate a function inside the page that uses parts of other functions, defined dynamically elsewhere. The code below is a minimal example / proof. In reality functionToInject() and otherFunctionToInject() are more complex and require the pages DOM. const browser = await puppeteer.launch(); const page = await browser.newPage(); await page.goto(someURL); var functionToInject = function(){ return 1+1; } var otherFunctionToInject = function
订阅 headless-browser