Pushing binary data to Amazon S3 using Node.js

Question

I\'m trying to take an image and upload it to an Amazon S3 bucket using Node.js. In the end, I want to be able to push the image up to S3, and then be able to access that S3

Answer 1

I believe you need to pass the content-length in the header as documented on the S3 docs: http://docs.aws.amazon.com/AmazonS3/latest/API/RESTObjectPUT.html

After spending quite a bit of time working on pushing assets to S3, I ended up using the AwsSum library with excellent results in production:

https://github.com/awssum/awssum-amazon-s3/

(See the documentation on setting your AWS credentials)

Example:

var fs = require('fs');
var bucket_name = 'your-bucket name'; // AwsSum also has the API for this if you need to create the buckets

var img_path = 'path_to_file';
var filename = 'your_new_filename';

// using stat to get the size to set contentLength
fs.stat(img_path, function(err, file_info) {

    var bodyStream = fs.createReadStream( img_path );

    var params = {
        BucketName    : bucket_name,
        ObjectName    : filename,
        ContentLength : file_info.size,
        Body          : bodyStream
    };

    s3.putObject(params, function(err, data) {
        if(err) //handle
        var aws_url = 'https://s3.amazonaws.com/' + DEFAULT_BUCKET + '/' + filename;
    });

});

UPDATE

So, if you are using something like Express or Connect which are built on Formidable, then you don't have access to the file stream as Formidable writes files to disk. So depending on how you upload it on the client side the image will either be in req.body or req.files. In my case, I use Express and on the client side, I post other data as well so the image has it's own parameter and is accessed as req.files.img_data. However you access it, that param is what you pass in as img_path in the above example.

If you need to / want to Stream the file that is trickier, though certainly possible and if you aren't manipulating the image you may want to look at taking a CORS approach and uploading directly to S3 as discussed here: Stream that user uploads directly to Amazon s3