Does AWS S3 GetObject provide random access?

时间秒杀一切 提交于 2020-05-17 05:45:06

问题


I can provide HTTP Range headers to AWS S3's GetObject to request a specified range of bytes of an object.

Is it truly random access, or does S3 have to process all of the object before that range before returning my requested range?

Is the range header simply reducing the bytes transferred, or does it also provide efficient random access?


回答1:


I did a quick test with a 2GB file and executed ranged gets for 8 bytes at various offsets in the file (including start, middle, and end). The total time seemed to be pretty consistent at 250ms user time (including starting node.js, loading packages, executing range GetObject), as measured by time from my Mac to us-east-1.

I wasn't able to find a definitive statement in the AWS documentation for the expected behavior here (though I'd hope and expect that it is close to O(1) constant time).

I'd encourage you to investigate further before committing to a design. And maybe update us here.

[Update] Here's the results of a slightly more extensive experiment. S3, Lambda, a 2gb file, and 100 reads of 100 bytes to random parts of the file:



来源:https://stackoverflow.com/questions/60176997/does-aws-s3-getobject-provide-random-access

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!