AWS S3 local server for integration testing

Question

I have some code that uploads and downloads files using AWS S3 (using the Java AWS SDK). I want to be able to write some tests for it, I was wondering if anyone has any good

Answer 1

Minio offers (in my opinion) the best set of features, flexibility and ease of use. It is available as a docker container or binary for major OSes.

To start with minio, it is as easy as:

• Download
• Start the binary minio server /data
• Use it

It works flawlessly with s3cmd and it has nice documentation for popular programming languages.

Answer 2

You can try localstack, which is an open source local AWS cloud stack made for testing. It provides implementations of several of AWS services, including S3.

It looks like a very popular open source project on GitHub.

Answer 3

I created different buckets to use for the different use cases. For example: my-dev-bucket and my-prod-bucket. I don't know if this meets your use criteria but you might want to consider it. The side benefit is it makes your pre production and production code follow the exact same flows.

Answer 4

You can try installing minio server on your laptop/system, its open source & single static binary. Server is S3 compatible. Then you can try minio-java client library for all operations, following is basic operations example.

Installing minio server [GNU/Linux]

$ wget https://dl.minio.io/server/minio/release/linux-amd64/minio
$ chmod 755 minio
$ ./minio --help  

Hope it helps Disclaimer: I work for Minio

Answer 5

I started a S3 Server API project for Ladon, it contains a simple File System Repository. Its a Java Project and contains a Spring Boot Starter for simple testing. Not all S3 API features are supported yet but I will add them on request. Its on Github: Ladon S3 Server

Answer 6

Late answer, will be useful mostly for Docker users. There's a great S3 compatible storage software called Riak CS and there's docker-riak-cs image that allows to quickly launch the server.

I've been using it for nearly 2 years for local development and integration testing with great success. It has some limitations, but nothing major that comes in the way, see api / compatibility documentation.

If you need Docker-less solution, you can set it up locally for each build, all setup and configuration scripts are available in docker-riak-cs repository.