Is there a server that provides an Amazon S3 style API locally? [closed]

Answer 1

I recently started using Skylable for my S3 needs, it's free (GPL). Their object storage supports replication, HA and deduplication and it's fully S3 compatible. You can run their software on a single server (iron, virtual machine or container) if you don't need redundancy or you can use more nodes if you need HA.

The number of replicas can be chosen per bucket, just like with Swift. I started with 2 nodes in replica 2 and added more nodes as our userbase started growing, to cope with the extra network traffic and the space requirements.

Adding more nodes is really easy and can be done on a live cluster.

In my experience Skylable proved to be faster and more reliable than Swift. It's written in C and OCaml, it's not interpreted. The memory footprint is really low, so I can run a node even on some cheap VPS.

Recently they announced to be working on Swift APIs, apparently their goal is to replace Swift.

Answer 2

While the original question is about S3 compatible software for (what it seems) production use, many are interested in the same software, but for local development and testing.

Regarding production-ready solution, there's a great S3 compatible storage software called Riak CS, it's solid and production proven for years, one drawback – it's not simple to setup. There are some limitations, but nothing major that gets in the way, see api / compatibility documentation.

Already a few great answers for development and testing, this will be useful for Docker users – there's docker-riak-cs image that allows to quickly launch Riak CS instance. I've been using it for nearly 2 years for local development and integration testing with great success.