Proxying file upload with node/express.js and node request results in corrupt files with almost dobble the file size

会有一股神秘感。 提交于 2021-02-11 17:01:33

问题


I'm trying to proxy our main server via a node.js/express.js server. I've got everything working as it should except file uploads. Files are uploaded and reach the server, however somewhere between the data being passed over to node.js request and the data reaching the server the file is corrupted.

A file with content-length 1833 ends up having a content-length 3274 once it reaches the server. I expect the file that reaches the server to stay at 1833. Running without the proxy works as expected.

This is the proxy POST endpoint

router.post("*", wrapAsync(async (req: any, res: any) => {
    const path = ROOT_PATH + req.originalUrl;
    logger.info("Proxy request to: POST " + path);
    const headers = JSON.parse(JSON.stringify(req.headers)); // create a copy of headers
    delete headers["content-length"]; // Without this node.js requests will set the content-length to null
    if (req.session.auth) {
        headers.Authorization = req.session.auth.Token;
    }
    const options = {
        agentOptions,
        body: req.rawBody,
        encoding: null as any,
        headers,
        json: false,
        resolveWithFullResponse: true,
        url: path
    };
    const result = await request.post(options);
    handleJSONResponse(result, res);
}));

The value for req.rawBody are created in the following little middleware:

app.use("/rest/api", (req: any, res, next) => {
    // This is a special piece of proxy middleware that provides the raw buffer
    req.rawBody = "";
    req.setEncoding("utf8");
    req.on("data", (chunk: any) => {
        req.rawBody += chunk;
    });
    req.on("end", () => {
        next();
    });
}, proxyRouter);

It is either something in the middleware or something in node.js request (either a bug or something in how I've configured it) that seems to be causing the issue. It is possible for me to handle the files in a separate endpoint so long as it gets POST:ed to the same endpoint in the server (ie, it does not have to be directly proxied, I have control what URLS the frontend uses)


回答1:


The issue was in the middleware

(req: any, res, next) => {
    // This is a special piece of proxy middleware that provides the raw buffer
    req.rawBody = "";
    req.setEncoding("utf8");
    req.on("data", (chunk: any) => {
        req.rawBody += chunk;
    });
    req.on("end", () => {
        next();
    });
}

Solved it by using bodyParser.raw() instead, ie:

app.use("/rest/api", bodyParsers.raw(), proxyRouter);


来源:https://stackoverflow.com/questions/60006391/proxying-file-upload-with-node-express-js-and-node-request-results-in-corrupt-fi

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!