How to transfer files between AWS s3 and AWS ec2

Question

I am using AWS ec2 instance. On this instance I\'m resulting some files. These operations are done by user data.

Now I want to store those files on s3 by writing co

Answer 1

Im assuming you need to copy from a new instance to s3. First create a IAM role so you dont need to run aws configure and this should all work at launch time. Second Install the cli and then define your copy job using aws cli in user data. Example below for Ubuntu 18. Assign the IAM role to your instance.

Userdata:

#!/bin/bash
apt-get update -y
apt install awscli -y
aws s3 cp *Path of data* s3://*destination bucket*  -recursive *--other options*

To create an IAM role 1. Go to the IAM console at https://console.aws.amazon.com/iam/ 2. In the left pane, select Roles then click Create role. 3. For Select type of trusted entity, choose AWS service. Select EC2. Select Next: Permissions. 4. For Attach permissions policies, choose the AWS managed policies that contain the required permissions or create a custom policy. 5. Click Service Choose A service, type S3 in Find a service box, click S3, select actions (all or read + write and others you may need) 6. Click on Resources, Select Resource (you can enter all resouces or limit to specific bucket with the ARN) 7. Click Next: Review policy. Enter an Name and Description. Click Create policy. 8. Return to Create Role page, click refresh, filter policy by name you assigned, select the policy. 9. Click Next: Tags and then add any required tags 10. On the Review page, enter a name and description for the role and click Create role.

References

• https://aws.amazon.com/premiumsupport/knowledge-center/ec2-instance-access-s3-bucket/
• https://aws.amazon.com/blogs/infrastructure-and-automation/amazon-s3-authenticated-bootstrapping-in-aws-cloudformation/
• https://optimalbi.com/aws-tips-and-tricks-moving-files-from-s3-to-ec2-instance/