boto3

What are the request per seconds of the AWS API Queries?

不想你离开。 提交于 2019-12-23 04:48:32
问题 What are the request per seconds and average response times(roundtrip) of the following API calls made by Boto-2.38/Boto3 ? conn=EC2(aws_access_key,aws_secret_key_id) Q1:images=conn.get_all_images(owners=['self']) Q2:instances=conn.get_only_instances() Q3:snapshots=conn.get_all_snapshots(owner='self') Q4:snapshot=conn.create_snapshot(volume_id, description) Q5:instance=conn.launch_instance(<params>) Q6:image=conn.create_image(instance_id,name,description) Q7:conn.deregister_image(image_id

How to submit Mechanical Turk ExternalQuestions with boto3

邮差的信 提交于 2019-12-23 03:00:17
问题 I am trying to programmatically create a question on mechanical turk using boto3, but I seem to be doing something wrong, since the ExternalQuestion data structure that is required for create_hit seems to be missing. I try to create the HIT like so: import boto3 #... client = boto3.client( 'mturk', endpoint_url=endpoint_url, region_name=region_name, aws_access_key_id=aws_access_key_id, aws_secret_access_key=aws_secret_access_key, ) question = ExternalQuestion(external_url=question_target,

AWS sqs queues don't list all queues

泪湿孤枕 提交于 2019-12-22 12:17:24
问题 I am new to AWS. I am following AWS tutorials quick start. I was following the AWS sqs code and sqs documentation. This is the following code which I wrote import boto3 from boto3.session import Session session = Session(aws_access_key_id="aswe343ffsf34r3fef3f", aws_secret_access_key="3423d23r2fwwfe232r2r", region_name="ap-southeast-1") sqs = session.resource("sqs") q_client = boto3.client("sqs") sqs.create_queue(QueueName='test-One', Attributes = {'DelaySeconds': '5' }) sqs.create_queue

boto3 cannot create client on pyspark worker?

只谈情不闲聊 提交于 2019-12-22 10:13:05
问题 I'm trying to send data from the workers of a Pyspark RDD to an SQS queue, using boto3 to talk with AWS. I need to send data directly from the partitions, rather than collecting the RDD and sending data from the driver. I am able to send messages to SQS via boto3 locally & from the Spark driver; also, I can import boto3 and create a boto3 session on the partitions. However when I try to create a client or resource from the partitions I receive an error. I believe boto3 is not correctly

How to use boto3 client with Python multiprocessing?

主宰稳场 提交于 2019-12-22 09:44:55
问题 Code looks something like this: import multiprocessing as mp from functools import partial import boto3 import numpy as np s3 = boto3.client('s3') def _something(**kwargs): # Some mixed integer programming stuff related to the variable archive return np.array(some_variable_related_to_archive) def do(s3): archive = np.load(s3.get_object('some_key')) # Simplified -- details not relevant pool = mp.pool() sub_process = partial(_something, slack=0.1) parts = np.array_split(archive, some_int)

Programmatically unset encryption for a file in aws s3

浪子不回头ぞ 提交于 2019-12-22 07:12:18
问题 I'm performing an android build via aws code build . The apk files generated are by default applied with server side encryption (aws-kms) I can unset the encryption manually by clicking as shown below from the s3 bucket by unselecting ASW-KMS which gives the following popup Here selecting None option manually will make the link downloadable. I want to achieve this programmatically . I have already tried adding permissions as mentioned here. Also did experiment a fair bit with python boto3.

boto3 searching unused security groups

让人想犯罪 __ 提交于 2019-12-22 05:53:15
问题 I am using AWS Python SDK Boto3 and I am trying to know which security groups are unused. With boto2 I did it but I do not know how to do the same with boto3. from boto.ec2.connection import EC2Connection from boto.ec2.regioninfo import RegionInfo import boto.sns import sys import logging from security_groups_config import config # Get settings from config.py aws_access_key = config['aws_access_key'] aws_secret_key = config['aws_secret_key'] ec2_region_name = config['ec2_region_name'] ec2

How can I add a tag to a key in boto (Amazon S3)?

跟風遠走 提交于 2019-12-22 05:39:17
问题 I am trying to tag a key that I've uploaded to S3. In the same below I just create a file from a string. Once I have they key, I'm not sure how to tag the file. I've tried Tag as well as TagSet. from boto.s3.bucket import Bucket from boto.s3.key import Key from boto.s3.tagging import Tag, TagSet k = Key(bucket) k.key = 'foobar/somefilename' k.set_contents_from_string('some data in file') Tag(k, 'the_tag') 回答1: As far as I can see in the docs, a setTags-method is only available on a bucket

How to determine what CloudFormation stack an AWS resource belongs to?

Deadly 提交于 2019-12-22 04:30:56
问题 Is there a reliable way in boto3 to determine what CloudFormation stack an AWS resource belongs to? Or if it belongs to a stack at all? Say I have a DynamoDB table or an EC2 instance, how do I find out what stack it is a member of? The boto3 API for CloudFormation gets pretty vague at the resource level, or so it appears. Any help is much appreciated. 回答1: You can pass PhysicalResourceId of a resource to desribe_stack_resources and get the stack information if it belongs to a CF stack To find

S3 giving me NoSuchKey error even when the key exists

痴心易碎 提交于 2019-12-22 04:02:18
问题 This is my boto3 command for getting the object with a specific key from an S3 bucket: resp = s3client.get_object(Bucket='<>-<>', Key='MzA1MjY1NzkzX2QudHh0') It gives the following error: botocore.errorfactory.NoSuchKey: An error occurred (NoSuchKey) when calling the GetObject operation: The specified key does not exist. I have checked in the bucket, and the key actually exists Did I miss something or did I do something wrong here? 回答1: You have a %0A at the end of your URL; that's a line