问题
Does anybody have a solution to sync a bucket between 2 different s3 providers?
For example one is Amazon S3, second is Wasabi S3?
That involves 2 different endpoints and 2 different sets of credentials.
Preferable without storage data locally first, talking about 1+ million files.
回答1:
This would not be possible.
In Amazon S3, it is possible to copy directly between two buckets, even in different regions, because the S3 services in each region communicate with each other.
This wouldn't be possible with a non-Amazon service.
Therefore, your best bet would be to launch an EC2 instance, use aws s3 sync to download the files, then use aws s3 sync (with a different endpoint & credentials) to connect to Wasabi.
It is possible that the S3-compatible providers might offer an easier method, but it appears that Wasabi recommends the 'download and copy' method: How do I copy files from AWS S3 to Wasabi? – Wasabi Knowledge Base
回答2:
There is this great open source tool "rclone" that can handle this.
It allows for sync between any two S3 compatible object stores - I used it for Minio to AWS S3
https://rclone.org/commands/rclone_sync/
来源：https://stackoverflow.com/questions/53381254/aws-cli-aws-sync-between-2-different-s3-providers