How to create a key as folder and value as files

问题

• I have a bucket name testfolder

• Inside testfolder there are test1,test2,test3

• Each folder have there are csv files

• Need to create a key value pair for folder and files

Expected out

output1 { 'test1':['csv1.csv'], 'test2':['csv2'], 'test3':['csv3']}

output2 { 'test1':'csv1.csv', 'test2':'csv2', 'test3':'csv3'}

#list all the objects
import boto3 
s3 = boto3.client("s3")
final_data = {}
all_objects = s3.list_objects(Bucket = 'testfolder') 
#List the object in subfolder
#create a dictionary

回答1:

You can check the following code which gives output1 format. For output2 you can modify the final_data accordingly:

from collections import defaultdict

import boto3

s3 = boto3.resource("s3")

final_data = defaultdict(list)

in_bucket = 'test-bucket-folder-4412313'

for obj in s3.Bucket(in_bucket).objects.all():
    print(obj.key)
    
    folder_name = obj.key.split('/')[0]
    file_name = obj.key.split('/')[1]

    final_data[folder_name].append(file_name)


print(dict(final_data))

It prints out:

{'test1': ['csv1.csv'], 'test2': ['csv2.csv'], 'test3': ['csv3.csv']}

来源：https://stackoverflow.com/questions/63358501/how-to-create-a-key-as-folder-and-value-as-files