Pipe a uploaded file to a remote server with node (ideally with same filename)

Question

I want to upload a file in my app.js server , which should pipe that file to a crossdomain server like my upload.js server.

The full code can be found under the foll

Answer 1

Updated

I believe you're missing the protocol from the url. It should work, if you add the http protocol to the url:

fs.createReadStream(file.path).pipe(request.post('http://localhost:4000/upload'))

Make Upload work

When you pipe the file contents to the POST function in upload.js, the multipart form data is lost. You need to create a new POST request and pass the original file contents.

Do the following in app.js:

 form.on('file', function(name, file) {

    var formData = {
      file: {
        value:  fs.createReadStream(file.path),
        options: {
          filename: file.originalFilename
        }
      }
    };

    // Post the file to the upload server
    request.post({url: 'http://localhost:4000/upload', formData: formData});
}

This will also pass the original filename. For more information see: https://github.com/request/request#multipartform-data-multipart-form-uploads

Answer 2

You actually can do stream.pipe(request.post(url));, and then on the another server your req variable will have a stream-like behavior, so you should be able to req.pipe(fs.createWriteStream(path));.

Use req.params on your route to preserve the filename.

Example code for server one (use some hashing like md5 to validate your request):

const stream = fs.createReadStream(path);

const md5 = crypto.createHash('md5')
    .update(`${filename}${secret}`)
    .digest('hex').toUpperCase();

const url = `http://${host}/upload/${md5}/${filename}`;

const r = request.post(url, (err, resp, body) => {});

stream.pipe(r);

Example code for server two:

router.post('/upload/:md5/:filename', function(req, res, next) {
    const { md5, filename } = req.params;

    const check = crypto.createHash('md5')
        .update(`${filename}${secret}`)
        .digest('hex').toUpperCase();

    if (md5 !== check) {
        res.writeHead(403);
        res.end('403');
        return;
    }

    req.pipe(fs.createWriteStream(filename));

    req.on('end', next);
});

Answer 3

I had a similar problem but I wanted to stream the file directly to the remote server instead of saving it locally first. Here's my modifications to get streaming to work:

app.post('/upload', function(request, response, next) {
    var form = new multiparty.Form();

    form.on('part', function(formPart) {
        var contentType = formPart.headers['content-type'];

        var formData = {
            file: {
                value: formPart,
                options: {
                    filename: formPart.filename,
                    contentType: contentType,
                    knownLength: formPart.byteCount
                }
            }
        };

        requestJs.post({
            url: 'http://localhost:4000/upload',
            formData: formData,

            // These may or may not be necessary for your server:
            preambleCRLF: true,
            postambleCRLF: true
        });
    });

    form.on('error', function(error) {
        next(error);
    });

    form.on('close', function() {
       response.send('received upload');
    });

    form.parse(request);
});