Search files(key) in s3 bucket takes longer time

跟風遠走 提交于 2019-12-11 04:17:56

问题


I have 10000 files in a s3 bucket.When I list all the files it takes 10 minutes. I want to implement a search module using BOTO (Python interface to AWS) which searches files based on user input. Is there a way I can search specific files with less time?


回答1:


AFAIK the best you can do is filter the results based on a file prefix using the prefix named parameter.




回答2:


There are two ways to implement the search...

Case 1. As suggested by john - you can specify the prefix of the s3 key file in your list method. that will return you result of S3 key files which starts with the given prefix.

Case 2. If you want to search the S3 key which are end with specific suffix or we can say extension then you can specify the suffix in delimiter. Remember it will give you correct result only in the case if you are giving suffix for the search item which is end with that string. Else delimiter is used for path separator.

I will suggest you Case 1 but if you want to faster search with specific suffix then you can try case 2



来源:https://stackoverflow.com/questions/12904326/search-fileskey-in-s3-bucket-takes-longer-time

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!