boto3

问题 I need to read multiple csv files from S3 bucket with boto3 in python and finally combine those files in single dataframe in pandas. I am able to read single file from following script in python s3 = boto3.resource('s3') bucket = s3.Bucket('test-bucket') for obj in bucket.objects.all(): key = obj.key body = obj.get()['Body'].read() Following is my path files/splittedfiles/Code-345678 In Code-345678 I have multiple csv files which I have to read and combine it to single dataframe in pandas

问题 I am using django-storage and boto3 for media and static files using aws s3. I need to get the object key of aws s3 bucket, so that I can generate a url for that object. client = boto3.client('s3') bucket_name = 'django-bucket' key = ??? u = client.generate_presigned_url('get_object', Params = {'Bucket': bucket_name, 'Key': key,'ResponseContentType':'image/jpeg', 'ResponseContentDisposition': 'attachment; filename="your-filename.jpeg"'}, ExpiresIn = 1000) These are in my settings: STATICFILES

问题 I'm reviewing IAM policies and roles that haven't been used in the last N number of days. In the console I can easily view recent usage under Access Advisor. I'd like to get the same in an automated way, but I can't find any documentation on getting this using CLI or SDK. Is this possible? 回答1: It is now available, check the link below https://aws.amazon.com/about-aws/whats-new/2018/12/iam_access_advisor_apis/ 回答2: Netflix has a tool called Aardvark to scrape the Access Advisor data from the

问题 I'm reviewing IAM policies and roles that haven't been used in the last N number of days. In the console I can easily view recent usage under Access Advisor. I'd like to get the same in an automated way, but I can't find any documentation on getting this using CLI or SDK. Is this possible? 回答1: It is now available, check the link below https://aws.amazon.com/about-aws/whats-new/2018/12/iam_access_advisor_apis/ 回答2: Netflix has a tool called Aardvark to scrape the Access Advisor data from the

问题 I want to upload and download files to S3 using boto3 without changing their "LastModified" date so I can keep tabs on the age of the contents. Whenever I upload or download a file it takes on the date of this operation and I lose the date that the contents were modified. I'm looking at the timestamp of the files using fileObj.get('LastModified') where the fileObj is taken from a paginator result. I'm using the following command to upload s3Client.upload_fileobj(data, bucket_name, destpath)

问题 I'm implementing Boto3 to upload files to S3, and all works fine. The process that I'm doing is the following: I get base64 image from FileReader Javascript object. Then I send the base64 by ajax to the server, I decode the base64 image and I generate a random name to rename the key argument data = json.loads(message['text']) dec = base64.b64decode(data['image']) s3 = boto3.resource('s3') s3.Bucket('bucket_name').put_object(Key='random_generated_name.png', Body=dec,ContentType='image/png',ACL

问题 I have simple lambda function that is located under following endpoint: https://******.execute-api.eu-west-2.amazonaws.com/lambda/add?x=1&y=2 AWS Chalice was used for adding simple endpoints here. @app.route('/{exp}', methods=['GET']) def add(exp): app.log.debug("Received GET request...") request = app.current_request app.log.debug(app.current_request.json_body) x = request.query_params['x'] y = request.query_params['y'] if exp == 'add': app.log.debug("Received ADD command...") result = int(x

问题 Edit : my original assumption was proved partly wrong. I've added a lengthy answer here on which I invite others to stress-test and correct. I am looking for a way to utilize the Boto3 S3 API in a single-threaded manner to mimic a threadsafe key-value store. In a nutshell, I want to use the calling thread rather than a new thread to do the upload. The default behavior of the .upload_fileobj() method in Boto3 (or .upload_file() ), as far as I can tell, is to kick the task off to a new thread

问题 The boto3 library provides several factory methods that returns resources. For example: dynamo = ( boto3 .resource('dynamodb') .Table(os.environ['DYNAMODB_TABLE']) ) I want to annotate those resources so I can get better type checking and completion, but the only type alike I could find was from boto3.dynamodb.table import TableResource . When I add that annotation: dynamo: TableResource = ( boto3 .resource('dynamodb') .Table(os.environ['DYNAMODB_TABLE']) ) The only method offered by auto

问题 How can I loop through all results in a DynamoDB query, if they span more than one page? This answer implies that pagination is built into the query function (at least in v2), but when I try this in v3, my items seem limited: import boto3 from boto3.dynamodb.conditions import Key, Attr dynamodb = boto3.resource('dynamodb') fooTable = dynamodb.Table('Foo') response = fooTable.query( KeyConditionExpression=Key('list_id').eq('123') ) count = 0 for i in response['Items']: count += 1 print count #