How can I get only one level of objects in a S3 bucket?

人盡茶涼 提交于 2021-02-18 05:41:29

问题


I want to list only the objects in a bucket that aren't buckets themselves. Is there a way of doing this short of parsing out the results of ListBucket?


回答1:


objects in a bucket that aren't buckets themselves

Buckets can't contain other buckets. Do you mean folders? S3 doesn't have a concept of folders either.

You can have 100 buckets per S3 account and each bucket can contain an unlimited number of objects/files. If you name your files with /'s in the filename, the AWS GUI tools (eg AWS Console, BucketExplorer etc) will interpret each section as a virtual folder. eg

A file named folder1/folder2/myfile.jpg will be stored in S3 as a 'flat' file with that name, but in the GUI tools it will appear as though a file named myfile.jpg is 2 subfolders down in folder1/folder2.

You can use the prefix and delimiter parameters to parse the results of a GET Bucket (List Objects) call. The same options are available in any of the SDKs too.

UPDATE to answer comment.

Assuming our S3 bucket looks like this:

mybucket
   folder1
      file1.txt
      file2.txt
      folder2
          file3.txt
          file4.txt
      folder3
          file5.txt
          file6.txt

Using prefix = "folder1/" would return all 6 files : file1.txt to file6.txt.

Using a prefix = "folder1/" and a delimiter = "/" would return 2 files:

    file1.txt
    file2.txt

And the CommonPrefixes collection of the response with contain

    folder1/folder2/
    folder1/folder3/



回答2:


There is a better approach using the latest boto3 version (1.14 as of now) and list_objects_v2 method.

import boto3

s3_client = boto3.client('s3')
response = s3_client.list_objects_v2(Bucket=bucket_name, Prefix=your_prefix)

print(response)



回答3:


I was searching for solving the same problem "To list first/root/x level directory only" and finally stumbled into something called "Paginator".

$results = $s3Client->getPaginator('ListObjects', [
    'Bucket'    => 'my-bucket',
    'Delimiter' => '/'
]);

$expression = '[CommonPrefixes[].Prefix, Contents[].Key][]';
foreach ($results->search($expression) as $item) {
    echo $item . "\n";
}

I tested it and works exactly as expected! :)

https://docs.aws.amazon.com/sdk-for-php/v3/developer-guide/guide_paginators.html



来源:https://stackoverflow.com/questions/5455284/how-can-i-get-only-one-level-of-objects-in-a-s3-bucket

标签
易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!