Downloading an entire S3 bucket?

Question

I noticed that there doesn\'t seem to be an option to download an entire S3 bucket from the AWS Management Console.

Is there an easy way to grab everything in one of

Answer 1

If you have only files there (no subdirectories) a quick solution is to select all the files (click on the first, Shift+click on the last) and hit Enter or right click and select Open. For most of the data files this will download them straight to your computer.

Answer 2

AWS CLI is the best option to download an entire S3 bucket locally.

1. Install AWS CLI.

2. Configure AWS CLI for using default security credentials and default AWS Region.

3. To download the entire S3 bucket use command

   aws s3 sync s3://yourbucketname localpath

Reference to use AWS cli for different AWS services: https://docs.aws.amazon.com/cli/latest/reference/

Answer 3

As @layke said, it is the best practice to download the file from the S3 cli it is a safe and secure. But in some cases, people need to use wget to download the file and here is the solution

aws s3 presign s3://<your_bucket_name/>

This will presign will get you temporary public URL which you can use to download content from S3 using the presign_url, in your case using wget or any other download client.

Answer 4

Another option that could help some OS X users is Transmit.

It's an FTP program that also let you connect to your S3 files. And, it has an option to mount any FTP or S3 storage as a folder in the Finder, but it's only for a limited time.

Answer 5

If you use Visual Studio, download "AWS Toolkit for Visual Studio".

After installed, go to Visual Studio - AWS Explorer - S3 - Your bucket - Double click

In the window you will be able to select all files. Right click and download files.

Answer 6

Use this command with the AWS CLI:

aws s3 cp s3://bucketname . --recursive