Extreme Sharding: One SQLite Database Per User

Question

I\'m working on a web app that is somewhere between an email service and a social network. I feel it has the potential to grow really big in the future, so I\'m concerned about

Answer 1

I am considering this same architecture as I basically wanted to use the server side SQLLIte databases as backup and synching copy for clients. My idea for querying across all the data is to use Sphinx for full-text search and run Hadoop jobs from flat dumps of all the data to Scribe and then expose the results as webservies. This post gives me some pause for thought however, so I hope people will continue to respond with their opinion.