Let me start out with a quick introduction to the architecture of a system I\'m considering migrating to S3+Cloudfront.
We have a number of entities order in a tree.
Based on popular request, I'm answering this question myself.
After gathering relevant metrics and doing some calculations, we ended up concluding we could live with less caching, offset by the faster object serving speed of CloudFront. The actual implementation is detailed on my blog: How to Set Up and Serve Private Content Using S3 and Amazon CloudFront