How can I capture all network requests and full response data when loading a page in Chrome?

Question

Using Puppeteer, I\'d like to load a URL in Chrome and capture the following information:

• request URL
• request headers
• request post data

Answer 1

I would suggest you to search for a quick proxy server which allows to write requests logs together with actual content.

The target setup is to allow proxy server to just write a log file, and then analyze the log, searching for information you need.

Don't intercept requests while proxy is working (this will lead to slow down)

The performance issues(with proxy as logger setup) you may encounter are mostly related to TLS support, please pay attention to allow quick TLS handshake, HTTP2 protocol in the proxy setup

E.g. Squid benchmarks show that it is able to process hundreds RPS, which should be enough for testing purposes