Puppeteer

Puppeteer wait page load after form submit

由 巧了我就是萌 提交于 2019-11-27 17:06:23
问题 I submit a form using the following code and i want Puppeteer to wait page load after form submit. await page.click("button[type=submit]"); //how to wait until the new page loads before taking screenshot? // i don't want this: // await page.waitFor(1*1000); //← unwanted workaround await page.screenshot({path: 'example.png'}); How to wait for page load with puppeteer? 回答1: You can wait for navigation asynchronously to avoid getting null on redirection, await Promise.all([ page.click("button

Puppeteer - Protocol error (Page.navigate): Target closed

由 微笑、不失礼 提交于 2019-11-27 16:47:45
问题 As you can see with the sample code below, I'm using Puppeteer with a cluster of workers in Node to run multiple requests of websites screenshots by a given URL: const cluster = require('cluster'); const express = require('express'); const bodyParser = require('body-parser'); const puppeteer = require('puppeteer'); async function getScreenshot(domain) { let screenshot; const browser = await puppeteer.launch({ args: ['--no-sandbox', '--disable-setuid-sandbox', '--disable-dev-shm-usage'] });

How to select an option from dropdown select

由 大憨熊 提交于 2019-11-27 14:35:56
I can click the selector but my question is how to select one of the options from the dropdown list? await page.click('#telCountryInput > option:nth-child(4)') Click the option using CSS selector does not work. For example, select a country code from a list like below, Puppeteer v0.13.0 has page.select() method, which does exactly that. You just have to give it the value to select. So, assuming you have an <option value="my-value"> in your <select> : await page.select('#telCountryInput', 'my-value') For dropdown component, I think we should consider 2 situations: native HTML select element

How to login in Puppeteer?

由 梦想与她 提交于 2019-11-27 13:28:29
问题 I'm new to javascript and puppeteer. I tried a login code below and failed. In comparison, I added pages2 and succeeded. How can I solved it? Thanks in advance. const CREDS = require('./creds'); async function main() { const puppeteer = require('puppeteer'); const browser = await puppeteer.launch({headless: false}); const page = await browser.newPage(); await page.setViewport({width: 1200, height: 720}) await page.goto('https://www.daum.net'); await page.waitForNavigation(); await page.type('

How to use puppeteer to dump WebSocket data

由 本小妞迷上赌 提交于 2019-11-27 13:07:59
问题 I want to get websocket data in this page https://upbit.com/exchange?code=CRIX.UPBIT.KRW-BTC, its websocket URL is dynamic and only valid during the first connection, the second time you connect to it it will not send data anymore. So I wonder that maybe headless chrome can help me to monitor the websocket data. Any ideas? Thanks! 回答1: You actually don't need to do anything complex on this. The URL though seems dynamic, but works fine through code as well. The reason it doesn't work is that

Puppeteer - scroll down until you can't anymore

由 僤鯓⒐⒋嵵緔 提交于 2019-11-27 12:18:17
问题 I am in a situation where new content is created when i scroll down. The new content has a specific class name. How can i keep scrolling down until all the elements has loaded? In other words, i want to reach the stage where if i keep scrolling down, nothing new will load. I was using code to scroll down, coupled with an await page.waitForSelector('.class_name'); The problem with this approach is that after all the elements have loaded, the code keeps on scrolling down, no new elements are

Puppeteer log inside page.evaluate

由 為{幸葍}努か 提交于 2019-11-27 11:12:11
问题 How can I console.log something inside the page.evaluate, passing it to node and using it during the evaluation of the page? I actually want to log the progress of the page.evaluate to the console and show some results to the user. 回答1: **Updated to work with puppeteer v1.4.x If all you want is to "log the progress of the page.evaluate to the console", then just const page = await browser.newPage(); page.on('console', consoleObj => console.log(consoleObj.text())); And use console.log in page

Running puppeteer with containerized chrome binary from another container

由 徘徊边缘 提交于 2019-11-27 08:37:15
问题 I want my code using puppeteer running in one container and using (perhaps by "executablePath" launch param?) a chrome binary from another container. Is this possible? any known solution for that? Use case: worker code runs in multiple k8 pods (as containers) . "Sometime" (might be often or not often) worker needs to run code utilizing puppeteer. I don't want to make the docker gigantic and limited as the puppeteer/chrome container is (1.5 GB If I recall correctly) I just want my code to be

Puppeteer: pass variable in .evaluate()

由 别说谁变了你拦得住时间么 提交于 2019-11-27 07:09:53
I'm trying to pass a variable into a page.evaluate() function in Puppeteer , but when I use the following very simplified example, the variable evalVar is undefined. I'm new to Puppeteer and can't find any examples to build on, so I need help passing that variable into the page.evaluate() function so I can use it inside. const puppeteer = require('puppeteer'); (async() => { const browser = await puppeteer.launch({headless: false}); const page = await browser.newPage(); const evalVar = 'WHUT??'; try { await page.goto('https://www.google.com.au'); await page.waitForSelector('#fbar'); const links

Unable to Run Multiple Node Child Processes without Choking on DigitalOcean

由 只愿长相守 提交于 2019-11-27 06:31:31
问题 I've been struggling to run multiple instances of Puppeteer on DigitalOcean for quite some time with little luck. I'm able to run ~5 concurrently using tools like puppeteer-cluster, but for some reason the whole thing just chokes with little helpful messaging. So, I switched to spawning ~5 child processes without any additional library -- just Puppeteer itself. Same issue. Chokes with no helpful errors. I'm able to run all of these jobs just fine locally, but after I deploy, I hit these walls
订阅 Puppeteer