How to upload small files to Amazon S3 efficiently in Python

Question

Recently, I need to implement a program to upload files resides in Amazon EC2 to S3 in Python as quickly as possible. And the size of files are 30KB.

I have tried some

Answer 1

I have the same problem as You. My solution was send the data to AWS SQS and then save them to S3 using AWS Lambda.

So data flow looks: app -> SQS -> Lambda -> S3

Entire process is asynchronous, but near real-time :)