Puppeteer

exposed function queryseldtcor not working in puppeteer

由 半腔热情 提交于 2020-05-22 07:39:36
问题 document.querySelectorAll('.summary').innerText; this throws error in below snippet saying "document.querySelector is not a function" in my puppeteer page's exposed fucntion "docTest" i want to pass specific node to each method and get the result insde the evaluate. same with document.getElemenetbyId const puppeteer = require('puppeteer'); //var querySelectorAll = require('query-selector'); let docTest = (document) => { var summary = document.querySelectorAll(.summary).innerText; console.log

While running the script throws cannot find module 'dotenv'

由 守給你的承諾、 提交于 2020-05-17 07:28:53
问题 While loading the .env file to pass env values to the getToken.js script in the cypress root folder throws Cannot find module 'dotenv'error. I have installed npm install dotenv . Could someone please advise what I am missing here ? .env file is available in cypress root folder. Environment : Windows 10 > git bash /command prompt const puppeteer = require("puppeteer"); require('dotenv').config({path: '.env'}) const baseURL = process.env.CYPRESS_BASE_URL const testsUser = process.env.CYPRESS

Fill input in .mhtml file opened in Browser

由 回眸只為那壹抹淺笑 提交于 2020-05-16 22:30:34
问题 I want to run e2e tests on mhtml files, i.e., fill out a form. Viewing and extracting data works really well on the mhtml file, but I cannot fill out any input fields (neither manually nor via puppeteer). You can try it out with this mhtml file I created from the StackOverflow login page. <input name="firstName" value=""> This is how I open the snapshot: browser = await puppeteer.launch() page = await browser.newPage() const snapshotPath = join('file://', __dirname, 'mysnapshot.mhtml') await

Puppeteer: how to listen for on innerHTML change

由 瘦欲@ 提交于 2020-05-15 04:54:06
问题 I have a chat app with status represented by this line: <span id='status'>Offline</span> and I want Puppeteer to log every time the text within this span changes. Say, in the beginning the status was "Offline", then it was changed to "Online", and then after some time to "Away" and so on. I want Puppeteer to capture those moments and the status (Offline>Online>Away) What I managed to do is the following: const page = await browser.newPage(); await page.goto('https://chat.com'); const

Puppeteer waitForSelector on multiple selectors

由 故事扮演 提交于 2020-05-13 04:12:10
问题 I have Puppeteer controlling a website with a lookup form that can either return a result or a "No records found" message. How can I tell which was returned? waitForSelector seems to wait for only one at a time, while waitForNavigation doesn't seem to work because it is returned using Ajax. I am using a try catch, but it is tricky to get right and slows everything way down. try { await page.waitForSelector(SELECTOR1,{timeout:1000}); } catch(err) { await page.waitForSelector(SELECTOR2); } 回答1:

How can I disable cache in puppeteer?

由 元气小坏坏 提交于 2020-05-11 06:33:25
问题 I want to disable cache in puppeteer, can anyone please tell me how I can do so? I found this page.setCacheEnabled(enabled) but I couldn't understand how to use the same. I am aware that the browser is launched without cache or cookies but in my case the browser is always running in the background thus need a different solution. 回答1: According to the puppeteer docs you can use await page.setCacheEnabled(enabled) This was added back in December. See Git Hub issue #1609 If you look at the

使用Puppeteer进行数据抓取(四)——图片下载

由 烈酒焚心 提交于 2020-04-30 11:31:10
大多数情况下，图片获取并不是很困难的事情，获取图片的url，然后模拟浏览器请求即可。但是，有的时候这种方法往往无法生效，常见的情形有： 动态图片，每次获取都是一个新的，例如图片验证码，重新获取时是一个新的验证码图片，已经失去了效果了。 动态上下文，有的网站为了反爬虫，获取图片时要加上其动态生成的cookie才行。 这些情况下，使用puppeteer驱动chrome浏览器能看到图片，但获取url后单独请求时，要么获取到的图片无效，要么获取不到图片。本文这里就简单的介绍下一些十分通用且有效的下载这些图片的方法。 截图： 截图是一种非常简单除暴的方法，大多数的时候也是最方便有效的。特别是对于验证码之类的动态生成的图片，这些验证码获取原始图片往往需要一定时间的分析，但chrome能直接截取渲染后生成的图片，直接跳过了分析过程，十分方便。 这里以 专利检索及分析 网为例，截取它登陆的验证码。 　　 首先用devtool分析其selector path。 发现其为 "#codePic" ，接下来的操作就非常简单了 await page . goto ( 'http://www.pss-system.gov.cn/sipopublicsearch/portal/uiIndex.shtml' ); const image = await page . waitForSelector ( '

PuppeteerSharp体验之旅

由 吃可爱长大的小学妹 提交于 2020-04-28 13:40:19
public static async Task<string> LogInAsync() { try { string ResultCookies = ""; //获取用户名 string UserName = Environment.UserName; var currentDirectory = Path.Combine(@"C:\Users\", UserName, @"AppData\Local\Google\Chrome\Application\", "Chrome.exe");//string currentDirectory = Path.GetDirectoryName(@"C:\Users\TT\AppData\Local\Google\Chrome\Application"); //指定Chrome.exe在这目录才行 if (!File.Exists(currentDirectory)) { currentDirectory = Path.GetDirectoryName(AppDomain.CurrentDomain.BaseDirectory); var downloadPath = Path.Combine(currentDirectory, "LocalChromium"); Console.WriteLine($"Attemping to set

How to pass a function in Puppeteers .evaluate() method?

由 ☆樱花仙子☆ 提交于 2020-04-28 00:14:38
问题 Whenever I try to pass a function, like this: var myFunc = function() { console.log("lol"); }; await page.evaluate(func => { func(); return true; }, myFunc); I get: (node:13108) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 1): Error: Evaluation failed: TypeError: func is not a function at func (<anonymous>:9:9) (node:13108) DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the

How to pass a function in Puppeteers .evaluate() method?

由 ＊爱你＆永不变心＊ 提交于 2020-04-28 00:10:08
问题 Whenever I try to pass a function, like this: var myFunc = function() { console.log("lol"); }; await page.evaluate(func => { func(); return true; }, myFunc); I get: (node:13108) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 1): Error: Evaluation failed: TypeError: func is not a function at func (<anonymous>:9:9) (node:13108) DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the
订阅 Puppeteer