问题
I have 2x S3 Amazon buckets
- Old stuff
- New stuff
and I wish to move a list of 10K or so items from the Old Stuff bucket (which is just a subset of the data in that bucket) to the New Stuff bucket.
I'm not sure of the best way to do this.
I was thinking of leveraging their REST API but nothing stood out that could do this. Secondly, I'm not sure that their API would handle bulk moving - so then I would need some suggestions about how to best fire off 10K odd REST api requests...
Any code examples would need to be preferred in .NET please.
Lastly, if someone suggests an Open Source library to do this.. can they please explain if the method(s) handle bulk requests .. and if not .. how can i handle so many requests in a short time.
回答1:
Is this just a one off move?
Why not use a GUI tool such Cloudberry Explorer or BucketExplorer. I'm pretty sure both can do parallel operations.
If you want or need to do it programmatically, you can use the AWS .NET SDK's CopyObject method to copy files between buckets. Then delete the original file to complete the move. You could wrap this up in a Parallel.For or any of the other built in libraries for parallel/async operations. See Task Parallel Library.
I guess there's nothing stopping you using these libraries to make multiple request to the REST API in parallel too.
来源：https://stackoverflow.com/questions/5702800/how-to-move-a-lot-of-items-from-one-s3-amazon-bucket-to-another