How can I obtain the original encoded response size when intercepting requests with Puppeteer?

问题

I'm using this code to log the encoded response size when loading a page in Chrome:

const puppeteer = require("puppeteer");

(async function() {
  const browser = await puppeteer.launch();
  const page = await browser.newPage();

  page._client.on("Network.loadingFinished", data => {
    console.log("finished", { encodedDataLength: data.encodedDataLength });
  });

  // await page.setRequestInterception(true);
  // page.on("request", async request => {
  //   request.continue();
  // });

  await page.goto("http://example.com");
  await browser.close();
})();

This is the output:

finished { encodedDataLength: 967 }

However, if I uncomment the four lines in the code snippet the output changes to:

finished { encodedDataLength: 0 }

This does make some sense, since the intercepted request could have been modified in some way by the client, and it would not have been gzipped again afterwards.

However, is there a way to access the original gzipped response size?

The Chrome trace also doesn't include the gzipped size:

"encodedDataLength": 0, "decodedBodyLength": 1270,

回答1:

We can use Content-Length header value for such case.

The good guys at google decided they won't fix some weird bugs closely related to encodedDataLength.

Check the code and result below to see proof.

page.on("request", async request => {
  request.continue();
});

// Monitor using _client
page._client.on("Network.responseReceived", ({ response }) => {
  console.log("responseReceived", [
    response.headers["Content-Length"],
    response.encodedDataLength
  ]);
});

page._client.on("Network.loadingFinished", data => {
  console.log("loadingFinished", [data.encodedDataLength]);
});

// Monitor using CDP
const devToolsResponses = new Map();
const devTools = await page.target().createCDPSession();
await devTools.send("Network.enable");

devTools.on("Network.responseReceived", event => {
  devToolsResponses.set(event.requestId, event.response);
});

devTools.on("Network.loadingFinished", event => {
  const response = devToolsResponses.get(event.requestId);
  const encodedBodyLength =
    event.encodedDataLength - response.headersText.length;
  console.log(`${encodedBodyLength} bytes for ${response.url}`);
});

Result without setRequestInterception:

responseReceived [ '606', 361 ]
loadingFinished [ 967 ]
606 bytes for http://example.com/

Result with setRequestInterception:

responseReceived [ '606', 0 ]
loadingFinished [ 0 ]
-361 bytes for http://example.com/

Tested with multiple gzip tool. Same result everywhere.

The Content-Length Header is far more reliable in every sense.

来源：https://stackoverflow.com/questions/52831249/how-can-i-obtain-the-original-encoded-response-size-when-intercepting-requests-w

标签

javascript

Puppeteer