Faster way to make S3 “folder hierarchy” than parsing of filenames?

烈酒焚心 提交于 2019-12-11 06:59:25

问题


I want to make a relatively basic tool to browse a bucket in S3 as a file hierarchy rather than simply a list of filenames with slashes in them.

Currently, I am using boto to get the list of keynames in a bucket and then parsing the keynames to make a nested dictionary of the "folders" and files. However, that process takes so long! Even just going through each key to get a list of all higher level folders takes 15+ minutes.

How do tools such as cyberduck give a list of folders so quickly?


回答1:


Check this link: http://docs.aws.amazon.com/AmazonS3/latest/dev/ListingKeysHierarchy.html

listObjects() has a parameter called delimiter, which could be set to / and resulting list of objects will look exactly as a tree of files. I think this is what you're looking for.



来源:https://stackoverflow.com/questions/17095192/faster-way-to-make-s3-folder-hierarchy-than-parsing-of-filenames

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!