Quick way to list all files in Amazon S3 bucket?

Question

I have an amazon s3 bucket that has tens of thousands of filenames in it. What\'s the easiest way to get a text file that lists all the filenames in the bucket?

Answer 1

Alternatively you can use Minio Client aka mc. Its Open Source and compatible with AWS S3. It is available for Linux, Windows, Mac, FreeBSD.

All you have do do is to run mc ls command for listing the contents.

$ mc ls s3/kline/
[2016-04-30 13:20:47 IST] 1.1MiB 1.jpg
[2016-04-30 16:03:55 IST] 7.5KiB docker.png
[2016-04-30 15:16:17 IST]  50KiB pi.png
[2016-05-10 14:34:39 IST] 365KiB upton.pdf

Note:

• s3: Alias for Amazon S3
• kline: AWS S3 bucket name

Installing Minio Client Linux Download mc for:

• 64-bit Intel from https://dl.minio.io/client/mc/release/linux-amd64/mc
• 32-bit Intel from https://dl.minio.io/client/mc/release/linux-386/mc
• 32-bit ARM from https://dl.minio.io/client/mc/release/linux-arm/mc

$ chmod 755 mc
$ ./mc --help

Setting up AWS credentials with Minio Client

$ mc config host add mys3 https://s3.amazonaws.com BKIKJAA5BMMU2RHO6IBB V7f1CwQqAcwo80UEIJEjc5gVQUSSx5ohQ9GSrr12

Note: Please replace mys3 with alias you would like for this account and ,BKIKJAA5BMMU2RHO6IBB, V7f1CwQqAcwo80UEIJEjc5gVQUSSx5ohQ9GSrr12 with your AWS ACCESS-KEY and SECRET-KEY

Hope it helps.

Disclaimer: I work for Minio