boto3

问题 I have an s3 bucket which has a large no of zip files having size in GBs. I need to calculate all zip files data length. I go through boto3 but didn't get it. I am not sure if it can directly read zip file or not but I have a process- Connect with the bucket. Read zip files from the bucket folder (Let's say folder is Mydata). Extract zip files to another folder named Extracteddata. Read Extracteddata folder and do action on files. Note: Nothing shouldn't download on local storage. All process

问题 The following loads an empty file to my s3 bucket. Why? df = pd.DataFrame([[1,2,3],[324,34,25], [463,23,43]]) out = df.to_csv(index=False) # unicode string (i'm in python 3) s3 = boto3.resource('s3') BUCKET = "MyBucketThatDefExistsAndIHaveAccessTo" bucket = s3.Bucket(BUCKET) obj = bucket.Object('TEST1') obj.put(df.to_csv(index=False)) # loads an empty file "TEST1" to my bucket. # I've also tried, but same result. obj.put(bytes(df.to_csv(index=False), 'utf8')) 回答1: Should it not be, obj.put

问题 I am trying to pass boto3 a list of bucket names and have it first enable versioning on each bucket, then enable a lifecycle policy on each. I have done aws configure, and do have two profiles, both current, active user profiles with all necessary permissions. The one I want to use is named "default." import boto3 # Create session s3 = boto3.resource('s3') # Bucket list buckets = ['BUCKET-NAME'] # iterate through list of buckets for bucket in buckets: # Enable Versioning bucketVersioning = s3

问题 I am trying to use boto3 and cloudsearchdomain but I am having troubles establishing some optional filters over my query. This is what I did: response = client.search( query=query, filterQuery= {'city':city}, partial=True, queryOptions= {'fields':'full_address'}, queryParser='simple', size=size) Based on the documentation of boto3, the filterQuery parameter should be an string, but I have no idea of the structure it should have and I found nothing on the internet. queryOptions should be a

问题 I have a data pipeline definition in json format, and I would like to 'put' that using Boto3 in Python. I know you can do this via the AWS CLI using put-pipeline-definition, but Boto3 (and the AWS API) use a different format, splitting the definition into pipelineObjects , parameterObjects and parameterValues . Do I need to write code to translate from a json definition to that expected by the API/Boto? If so, is there a library that does this? 回答1: The AWS CLI has code that does this

问题 I have a very simple sandbox I'm trying to get to work so I can use it in a bigger application: ec2_client = boto3.client( 'ec2', aws_access_key_id=ACCESS_KEY, aws_secret_access_key=SECRET_KEY) response = ec2_client.describe_instances() print(response) and it results in... { 'Reservations': [], 'ResponseMetadata': { 'RequestId': '2c28e8aa-da6d-4ca4-8ea7-f672518cac9f', 'HTTPStatusCode': 200, 'HTTPHeaders': { 'content-type': 'text/xml;charset=UTF-8', 'transfer-encoding': 'chunked', 'vary':

问题 I have an API Gateway with Lambdas behind, for some of the endpoints I want to schedule an execution in the future, to run once, for example the REST call was made at T time, I want that lambda to schedule an execution ONCE at T+20min. The only solution I found to achieve this is to use boto3 and Cloudwatch to setup a cron at the moment the REST call was made, send an event with the payload, then when the delayed lambda runs, it removes the rule. I found this very heavy, is there any other

问题 I'm working with AWS Lambda and I would like to make a simple query in athena and store my data in an s3. My code : import boto3 def lambda_handler(event, context): query_1 = "SELECT * FROM test_athena_laurent.stage limit 5;" database = "test_athena_laurent" s3_output = "s3://athena-laurent-result/lambda/" client = boto3.client('athena') response = client.start_query_execution( QueryString=query_1, ClientRequestToken='string', QueryExecutionContext={ 'Database': database },

问题 I have a script which monitors a directory using inotify-tools . When a new file is added to the directory, the script calls a python program and passes the path and fill name as arguments. It all works fine, except that I get a ModuleNotFoundError ... the module in question is boto3 (for AWS S3). The python script works fine in Pycharms, so I really do not know what is causing the error. Monitor script: source=$1 pythonscriptlocation=$2 inotifywait -m $source -e create -e moved_to | while

问题 I am trying to download files from a s3 bucket by using the Access Key ID and Secret Access Key provided by https://db.humanconnectome.org. However, even though I am able to navigate the database and find the files (as I have configured my credentials via aws cli), attempting to download them results in the following error: "botocore.exceptions.ClientError: An error occurred (403) when calling the HeadObject operation: Forbidden" With the same credentials, I can browse the same database and