I was wondering if anyone knew what exactly an s3 prefix was and how it interacts with amazon\'s published s3 rate limits:
Amazon S3 automatically sca
The upvoted answer on this was a bit misleading for me. If these are the paths
bucket/folder1/sub1/file
bucket/folder1/sub2/file
bucket/1/file
bucket/2/file
Your prefix for file would actually be
folder1/sub1/
folder1/sub2/
1/file
2/file
https://docs.aws.amazon.com/AmazonS3/latest/dev/ListingKeysHierarchy.html Please se docs. I had issues with the leading '/' when trying to list keys with the airflow s3hook.