How to detect the request come from Puppeteer?

北战南征 提交于 2020-11-29 03:10:23

问题


I wonder whether or not exists some flag or tag that the website can use it to detect the request came from Puppeteer?

When I ran my code based on Puppeteer to visit the target website, I found that the website seems to know the request was made by Puppeteer.

How can it do?


回答1:


If you are running the puppeteer and would like to pass some information to the website to catch your crawling, the best way to do so would be to set a custom user agent:

const browser = await puppeteer.launch({
    args: ['--user-agent=hhh'],
});
const page = await browser.newPage();

See here more info

Viceversa, if you own a website and would like to know if the visits are real or from a bot (puppeteer, a scraper, or anything else) see this answer for some of them. Also this answer




回答2:


I found a way to cross the limitation. It's an easy way:

const browser = await puppeteer.launch({headless: false, ignoreDefaultArgs: ["--enable-automation"],});

This will let the browser to not setup navigator.webdriver variable.



来源:https://stackoverflow.com/questions/58567713/how-to-detect-the-request-come-from-puppeteer

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!