boto3

Amazon AWS S3 browser-based upload using POST -

南楼画角 提交于 2020-01-01 09:06:08
问题 I am building a web application that includes a file upload feature. My goal is to initiate upload from users directly to an S3 bucket. The strategy is to pre-sign a POST request that will get submitted as a form. The roadblock is a SignatureDoesNotMatch error - as far as I can tell I've conformed to the documentation, and have explored a lot of options, but still unable to resolve. I am able to generate presigned download links. Referencing: AWS POST documentation Example boto3 generate

put_records() only accepts keyword arguments in Kinesis boto3 Python API

夙愿已清 提交于 2020-01-01 03:34:11
问题 from __future__ import print_function # Python 2/3 compatibility import boto3 import json import decimal #kinesis = boto3.resource('kinesis', region_name='eu-west-1') client = boto3.client('kinesis') with open("questions.json") as json_file: questions = json.load(json_file) Records = [] count = 0 for question in questions: value1 = question['value'] if value1 is None: value1 = '0' record = { 'StreamName':'LoadtestKinesis', 'Data':b'question','PartitionKey':'value1' } Records.append(record)

Send an SMS via AWS SNS using boto3 in an AWS Lambda function?

喜欢而已 提交于 2019-12-31 10:39:52
问题 I would like to send an SMS message from an AWS Lambda function using the boto3 publish method to notify the user of issues via SMS. My lambda function is written in Python and I am using the boto3 module. My lambda function has full rights to SNS. I have this code, sns = boto3.client('sns') sns.publish( PhoneNumber = '+11234567890', Message = 'Simple text message' ) According to the boto3 documentation, the publish method accepts the following parameters, response = client.publish( TopicArn=

s3 - how to get fast line count of file? wc -l is too slow

我是研究僧i 提交于 2019-12-31 03:35:11
问题 Does anyone have a quick way of getting the line count of a file hosted in S3? Preferably using the CLI, s3api but I am open to python/boto as well. Note: solution must run non-interactively, ie in an overnight batch. Right no i am doing this, it works but takes around 10 minutes for a 20GB file: aws cp s3://foo/bar - | wc -l 回答1: Here's two methods that might work for you... Amazon S3 has a new feature called S3 Select that allows you to query files stored on S3. You can perform a count of

django boto3: NoCredentialsError — Unable to locate credentials

孤者浪人 提交于 2019-12-30 19:56:41
问题 I am trying to use boto3 in my django project to upload files to Amazon S3. Credentials are defined in settings.py : AWS_ACCESS_KEY = xxxxxxxx AWS_SECRET_KEY = xxxxxxxx S3_BUCKET = xxxxxxx In views.py : import boto3 s3 = boto3.client('s3') path = os.path.dirname(os.path.realpath(__file__)) s3.upload_file(path+'/myphoto.png', S3_BUCKET, 'myphoto.png') The system complains about Unable to locate credentials . I have two questions: (a) It seems that I am supposed to create a credential file ~/

How to execute commands on AWS Instance using Boto3

半世苍凉 提交于 2019-12-30 18:00:58
问题 Can anyone tell me if we can execute Shell Commands using Boto3 on Launched AWS instance. I read at few places about "boto.manage.cmdshell" but it is deprecated in Boto3. Appreciate any help. Regards, Saurabh 回答1: ssm_client = boto3.client('ssm') response = ssm_client.send_command( InstanceIds=['i-03#####'], DocumentName="AWS-RunShellScript", Parameters={'commands': ['start ecs']}, ) command_id = response['Command']['CommandId'] output = ssm_client.get_command_invocation( CommandId=command_id

How to execute commands on AWS Instance using Boto3

余生长醉 提交于 2019-12-30 17:59:21
问题 Can anyone tell me if we can execute Shell Commands using Boto3 on Launched AWS instance. I read at few places about "boto.manage.cmdshell" but it is deprecated in Boto3. Appreciate any help. Regards, Saurabh 回答1: ssm_client = boto3.client('ssm') response = ssm_client.send_command( InstanceIds=['i-03#####'], DocumentName="AWS-RunShellScript", Parameters={'commands': ['start ecs']}, ) command_id = response['Command']['CommandId'] output = ssm_client.get_command_invocation( CommandId=command_id

What is the difference between boto3 list_objects and list_objects_v2?

房东的猫 提交于 2019-12-29 05:09:26
问题 I'm trying to list objects in an Amazon s3 bucket in python using boto3 . It seems boto3 has 2 functions for listing the objects in a bucket: list_objects() and list_objects_v2() . What is the difference between the 2 and what is the benefit of using one over the other? 回答1: Comparison side by side. list_objects() : response = client.list_objects( Bucket='string', Delimiter='string', EncodingType='url', #Marker to list continuous page Marker='string', MaxKeys=123, Prefix='string' ) list

How to SSH and run commands in EC2 using boto3?

╄→尐↘猪︶ㄣ 提交于 2019-12-28 06:00:22
问题 I want to be able to ssh into an EC2 instance, and run some shell commands in it, like this. How do I do it in boto3? 回答1: You can use the following code snippet to ssh to an EC2 instance and run some command from boto3. import boto3 import botocore import paramiko key = paramiko.RSAKey.from_private_key_file(path/to/mykey.pem) client = paramiko.SSHClient() client.set_missing_host_key_policy(paramiko.AutoAddPolicy()) # Connect/ssh to an instance try: # Here 'ubuntu' is user name and 'instance

VPC-enabled Lambda function cannot launch/access EC2 in the same VPC

泄露秘密 提交于 2019-12-25 17:46:26
问题 I have a VPC enabled Lambda function which attempts to launch an EC2 using a launch template. The EC2 launch step (run_instances) fails with the below generic network error. Calling the invoke API action failed with this message: Network Error I can launch an instance successfully directly from the launch template, so I think everything is fine with the launch template. I have configured the following in the launch template Amazon Machine Image ID Instance type Key Pair A network interface