Stream file uploaded with Express.js through gm to eliminate double write

Question

I'm using Express.js and have a route to upload images that I then need to resize. Currently I just let Express write the file to disk (which I think uses node-formidable under the covers) and then resize using gm (http://aheckmann.github.com/gm/) which writes a second version to disk.

gm(path)
  .resize(540,404)
  .write(dest, function (err) { ... });

I've read that you can get a hold of the node-formidable file stream before it writes it to disk, and since gm can accept a stream instead of just a path, I should be able to pass this right through eliminating the double write to disk.

I think I need to override form.onPart but I'm not sure where (should it be done as Express middleware?) and I'm not sure how to get a hold of form or what exactly to do with the part. This is the code skeleton that I've seen in a few places:

form.onPart = function(part) {
    if (!part.filename) { form.handlePart(part); return; }

    part.on('data', function(buffer) {

    });
    part.on('end', function() {

    }
}

Can somebody help me put these two pieces together? Thanks!

Answer 1

You're on the right track by rewriting form.onPart. Formidable writes to disk by default, so you want to act before it does.

Parts themselves are Streams, so you can pipe them to whatever you want, including gm. I haven't tested it, but this makes sense based on the documentation:

var form = new formidable.IncomingForm;
form.onPart = function (part) {
  if (!part.filename) return this.handlePart(part);

  gm(part).resize(200, 200).stream(function (err, stdout, stderr) {
    stdout.pipe(fs.createWriteStream('my/new/path/to/img.png'));
  });
};

As for the middleware, I'd copypaste the multipart middleware from Connect/Express and add the onPart function to it: http://www.senchalabs.org/connect/multipart.html

It'd be a lot nicer if formidable didn't write to disk by default or if it took a flag, wouldn't it? You could send them an issue.