Puppeteer

How do i return a value from page.evaluate() in puppeteer?

由 早过忘川 提交于 2020-01-13 20:48:10
问题 I am trying to get a value from inside page.evaluate() body in my YouTube scraper that I've built using Puppeteer. I am unable to return the result from page.evaluate() . How do I achieve this? Here's the code: let boxes2 = [] const getData = async() => { return await page.evaluate(async () => { // scroll till there's no more room to scroll or you get at least 250 boxes console.log(await new Promise(resolve => { var scrolledHeight = 0 var distance = 100 var timer = setInterval(() => { boxes =

Puppeteer get request redirects

由 独自空忆成欢 提交于 2020-01-13 19:27:47
问题 Is there any way to use puppeteer to get the redirects with the response body (if there are any) of the request? I implemented the following code but I can't find a way to get the redirects... const page = await browser.newPage() page.on('request', (data) => console.log(data)); await page.on('response', response => { const url = response.url(); response.buffer() .then ( buffer => { bufferString = buffer.toString(); }, error => { console.log(error) } ) }) await page.goto('https://www.ford.com'

Puppeteer get request redirects

由 怎甘沉沦 提交于 2020-01-13 19:27:13
问题 Is there any way to use puppeteer to get the redirects with the response body (if there are any) of the request? I implemented the following code but I can't find a way to get the redirects... const page = await browser.newPage() page.on('request', (data) => console.log(data)); await page.on('response', response => { const url = response.url(); response.buffer() .then ( buffer => { bufferString = buffer.toString(); }, error => { console.log(error) } ) }) await page.goto('https://www.ford.com'

Puppeteer get request redirects

由 蹲街弑〆低调 提交于 2020-01-13 19:27:06
问题 Is there any way to use puppeteer to get the redirects with the response body (if there are any) of the request? I implemented the following code but I can't find a way to get the redirects... const page = await browser.newPage() page.on('request', (data) => console.log(data)); await page.on('response', response => { const url = response.url(); response.buffer() .then ( buffer => { bufferString = buffer.toString(); }, error => { console.log(error) } ) }) await page.goto('https://www.ford.com'

How to use imported function inside page.evaluate in Puppeteer with Jest?

由 China☆狼群 提交于 2020-01-13 09:16:20
问题 I have a TypeScript project that uses Jest for unit tests and have just added Puppeteer to the mix with the intention to run some tests on the client. It works fine, unless I try use imported functions inside page.evaluate . For example, I have the following in HdpiCanvas.test.ts : import { createHdpiCanvas } from "./HdpiCanvas"; test("createHdpiCanvas", async () => { await page.setViewport({ width: 800, height: 600, deviceScaleFactor: 2 }); let size = await page.evaluate(() => { const canvas

Node.js puppeteer - How to set navigation timeout?

由 萝らか妹 提交于 2020-01-12 11:44:14
问题 I'm using node.js and puppeteer to get some data. Some of the files I'm opening are quite large ... and then I get an error: Error: our error { TimeoutError: Navigation Timeout Exceeded: 30000ms exceeded at Promise.then (/project/node_modules/puppeteer/lib/NavigatorWatcher.js:74:21) at <anonymous> name: 'TimeoutError' } How can I ignore it or set a higher timeout? That's my script: await page.goto('url'+tableCell04Val, {waitUntil: 'load'}); 回答1: You can use timeout: 0 to disabled timeout

Node.js puppeteer - How to set navigation timeout?

由 孤者浪人 提交于 2020-01-12 11:40:40
问题 I'm using node.js and puppeteer to get some data. Some of the files I'm opening are quite large ... and then I get an error: Error: our error { TimeoutError: Navigation Timeout Exceeded: 30000ms exceeded at Promise.then (/project/node_modules/puppeteer/lib/NavigatorWatcher.js:74:21) at <anonymous> name: 'TimeoutError' } How can I ignore it or set a higher timeout? That's my script: await page.goto('url'+tableCell04Val, {waitUntil: 'load'}); 回答1: You can use timeout: 0 to disabled timeout

Puppeteer: How to listen to a specific response?

由 ε祈祈猫儿з 提交于 2020-01-12 07:44:06
问题 I'm tinkering with the headless chrome node api called puppeteer . I'm wondering how to listen to a specific request response and how to act in consequence. I have look at events requestfinish and response but it gives me all the request/responses already performed in the page. How can I achieve commented behaviour? Thanks ! 回答1: One option is to do the following: page.on('response', response => { if (response.url().endsWith("your/match")) console.log("response code: ", response.status()); //

How to delete existing text from input using Puppeteer?

由 拟墨画扇 提交于 2020-01-12 06:59:27
问题 I'm trying to test amending text in an editable input which contains the title of the current record - and I want to able to test editing such text, replacing it with something else. I know I can use await page.type('#inputID', 'blah'); to insert "blah" into the textbox (which in my case, having existing text, only appends "blah"), however, I cannot find any page methods1 that allow deleting or replacing existing text. 回答1: You can use page.evaluate to manipulate DOM as you see fit: await

轻松搭建基于 Serverless 的文档图片在线转换服务

由 十年热恋 提交于 2020-01-08 14:23:44
【推荐】2019 Java 开发者跳槽指南.pdf(吐血整理) >>> 函数计算（Function Compute） ： 函数计算 是一个事件驱动的服务，通过函数计算，用户无需管理服务器等运行情况，只需编写代码并上传。函数计算准备计算资源，并以弹性伸缩的方式运行用户代码，而用户只需根据实际代码运行所消耗的资源进行付费。函数计算更多信息 参考 。 1 核心优势 异构服务：支持多种运行环境 用于媒体文件转换的库或者二进制往往和业务的运行环境不同，对计算资源的需求也有较大的差异。 函数计算支持多种运行环境，支持为独立的转换函数定制算力单位，根据请求量自动伸缩，让核心业务和支持业务做到较好的分离。 灵活触发：支持多种触发方式 函数计算既可以通过集成事件源服务（OSS、API 网关、日志服务和表格存储）产生事件来触发函数执行，也可以通过 HTTP 触发器使用 HTTP 请求触发函数执行，还支持 API/SDK 直接调用函数。 弹性伸缩: 轻松应对负载的波峰波谷 函数计算提供跨可用区的容灾能力，根据请求量自动进行毫秒级弹性扩容，快速调度计算资源，轻松应对业务洪峰。 预留实例功能彻底消除实例冷启动带来的延时毛刺，为在线应用迁移至函数计算扫清障碍! 工程效率：搭建速度快，运维代价小 使用函数计算，开发者无需管理服务器等基础设施，只需编写并上传代码，函数计算准备好计算资源，弹性可靠地运行任务
订阅 Puppeteer