How to get contents of a text file from AWS s3 using a lambda function?

Question

I was wondering if I could set up a lambda function for AWS, triggered whenever a new text file is uploaded into an s3 bucket. In the function, I would like to get the conte

Answer 1

The S3 object key and bucket name are passed into your Lambda function via the event parameter. You can then get the object from S3 and read its contents.

Basic code to retrieve bucket and object key from the Lambda event is as follows:

exports.handler = function(event, context, callback) {
   var src_bkt = event.Records[0].s3.bucket.name;
   var src_key = event.Records[0].s3.object.key;
};

Once you have the bucket and key, you can call getObject to retrieve the object:

var AWS = require('aws-sdk');
var s3 = new AWS.S3();

exports.handler = function(event, context, callback) {

    // Retrieve the bucket & key for the uploaded S3 object that
    // caused this Lambda function to be triggered
    var src_bkt = event.Records[0].s3.bucket.name;
    var src_key = event.Records[0].s3.object.key;

    // Retrieve the object
    s3.getObject({
        Bucket: src_bkt,
        Key: src_key
    }, function(err, data) {
        if (err) {
            console.log(err, err.stack);
            callback(err);
        } else {
            console.log("Raw text:\n" + data.Body.toString('ascii'));
            callback(null, null);
        }
    });
};

Here's an updated JavaScript example using ES6-style code and promises, minus error-handling:

const AWS = require('aws-sdk');
const s3 = new AWS.S3();

exports.handler = async (event, context) => {
  const Bucket = event.Records[0].s3.bucket.name;
  const Key = event.Records[0].s3.object.key;
  const data = await s3.getObject({ Bucket, Key }).promise();
  console.log("Raw text:\n" + data.Body.toString('ascii'));
};

A number of posters have asked for the equivalent in Java, so here's an example:

package example;

import java.net.URLDecoder;

import com.amazonaws.services.lambda.runtime.Context;
import com.amazonaws.services.lambda.runtime.RequestHandler;
import com.amazonaws.services.lambda.runtime.events.S3Event;
import com.amazonaws.services.s3.AmazonS3;
import com.amazonaws.services.s3.AmazonS3Client;
import com.amazonaws.services.s3.event.S3EventNotification.S3EventNotificationRecord;

public class S3GetTextBody implements RequestHandler<S3Event, String> {

    public String handleRequest(S3Event s3event, Context context) {
        try {
            S3EventNotificationRecord record = s3event.getRecords().get(0);

            // Retrieve the bucket & key for the uploaded S3 object that
            // caused this Lambda function to be triggered
            String bkt = record.getS3().getBucket().getName();
            String key = record.getS3().getObject().getKey().replace('+', ' ');
            key = URLDecoder.decode(key, "UTF-8");

            // Read the source file as text
            AmazonS3 s3Client = new AmazonS3Client();
            String body = s3Client.getObjectAsString(bkt, key);
            System.out.println("Body: " + body);
            return "ok";
        } catch (Exception e) {
            System.err.println("Exception: " + e);
            return "error";
        }
    }
}

Answer 2

I am using lambda function with a python 3.6 environment. The code below will read the contents of a file main.txt inside bucket my_s3_bucket. Make sure to replace name of bucket and file name according to your needs.

def lambda_handler(event, context):
    # TODO implement
    import boto3

    s3 = boto3.client('s3')
    data = s3.get_object(Bucket='my_s3_bucket', Key='main.txt')
    contents = data['Body'].read()
    print(contents)

Answer 3

You can use data.Body.toString('ascii') to get the contents of the text file, assuming that the text file was encoded used ascii format. You can also pass other encoding types to the function. Check out Node-Buffer for further details.