amazon-web-services

boto3 - AWS lambda -copy files between buckets

点点圈 提交于 2021-02-07 14:17:12
问题 I am trying to copy multiple files in a source bucket to a destination bucket using AWS lambda and am getting the error below. Bucket structures are as follows Source Buckets mysrcbucket/Input/daily/acctno_pin_xref/ABC_ACCTNO_PIN_XREF_FULL_20170926_0.csv.gz mysrcbucket/Input/daily/acctno_pin_xref/ABC_ACCTNO_PIN_XREF_FULL_20170926_1.csv.gz mysrcbucket/Input/daily/acctno_pin_xref/ABC_ACCTNO_PIN_XREF_count_20170926.inf Destination Buckets mydestbucket/Input/daily/acctno_pin_xref/ABC_ACCTNO_PIN

How S3 select pricing works? What is data returned and scanned in s3 select means

左心房为你撑大大i 提交于 2021-02-07 13:40:38
问题 I have a 1M rows of CSV data. select 10 rows, Will I be billed for 10 rows. What is data returned and data scanned means in S3 Select? There is less documentation on these terms of S3 select 回答1: To keep things simple lets forget for some time that S3 reads in a columnar way. Suppose you have the following data: | City | Last Updated Date | |------------|---------------------| | London | 1st Jan | | London | 2nd Jan | | New Delhi | 2nd Jan | A query for fetching the latest update date forces

DynamoDB The security token included in the request is invalid UnrecognizedClientException

。_饼干妹妹 提交于 2021-02-07 13:34:22
问题 I have ~/.aws/credentials and config files and my code is below clientConfiguration.setProxyHost("MYPROXY"); clientConfiguration.setProxyPort(port); clientConfiguration.setProxyUsername("username"); clientConfiguration.setProxyPassword("pw"); clientConfiguration.setPreemptiveBasicProxyAuth(false); AmazonDynamoDBClient client = new AmazonDynamoDBClient(new ProfileCredentialsProvider("MY_PROFILE"),clientConfiguration); //client.withRegion(Regions.US_EAST_1); DynamoDBMapper mapper = new

DynamoDB The security token included in the request is invalid UnrecognizedClientException

感情迁移 提交于 2021-02-07 13:30:49
问题 I have ~/.aws/credentials and config files and my code is below clientConfiguration.setProxyHost("MYPROXY"); clientConfiguration.setProxyPort(port); clientConfiguration.setProxyUsername("username"); clientConfiguration.setProxyPassword("pw"); clientConfiguration.setPreemptiveBasicProxyAuth(false); AmazonDynamoDBClient client = new AmazonDynamoDBClient(new ProfileCredentialsProvider("MY_PROFILE"),clientConfiguration); //client.withRegion(Regions.US_EAST_1); DynamoDBMapper mapper = new

DynamoDB The security token included in the request is invalid UnrecognizedClientException

六月ゝ 毕业季﹏ 提交于 2021-02-07 13:30:24
问题 I have ~/.aws/credentials and config files and my code is below clientConfiguration.setProxyHost("MYPROXY"); clientConfiguration.setProxyPort(port); clientConfiguration.setProxyUsername("username"); clientConfiguration.setProxyPassword("pw"); clientConfiguration.setPreemptiveBasicProxyAuth(false); AmazonDynamoDBClient client = new AmazonDynamoDBClient(new ProfileCredentialsProvider("MY_PROFILE"),clientConfiguration); //client.withRegion(Regions.US_EAST_1); DynamoDBMapper mapper = new

Sending file direct from browser to S3 but changing file name

∥☆過路亽.° 提交于 2021-02-07 13:18:35
问题 I am using signed authorized S3 uploads so that users can upload files directly from their browser to S3 bypassing my server. This presently works, but the file name is the same as on the user's machine. I'd like to save it on S3 as a different name. The formdata I post to amazon looks like this: var formData = new FormData(); formData.append('key', targetPath); // e.g. /path/inside/bucket/myFile.mov formData.append('AWSAccessKeyId', s3Auth.AWSAccessKeyId); // aws public key formData.append(

Sending file direct from browser to S3 but changing file name

十年热恋 提交于 2021-02-07 13:17:09
问题 I am using signed authorized S3 uploads so that users can upload files directly from their browser to S3 bypassing my server. This presently works, but the file name is the same as on the user's machine. I'd like to save it on S3 as a different name. The formdata I post to amazon looks like this: var formData = new FormData(); formData.append('key', targetPath); // e.g. /path/inside/bucket/myFile.mov formData.append('AWSAccessKeyId', s3Auth.AWSAccessKeyId); // aws public key formData.append(

CloudFormation nested stack name

醉酒当歌 提交于 2021-02-07 12:51:09
问题 I need to set nested stack name explicitly in a CloudFormation template, but don't see such option in AWS documentation. Is there way to achieve this? I can specify stack name, when running a parent stack, but all nested stacks, got a randomly generated stack name, based on a resource name created, like: VPC: Type: AWS::CloudFormation::Stack Properties: TemplateURL: https://s3-eu-west-1.amazonaws.com/cf-templates-wtmg/vpc.yaml Parameters: EnvironmentName: !Ref AWS::StackName Which will

AWS SDK for .NET can't access credentials with IIS

二次信任 提交于 2021-02-07 12:51:05
问题 I'm having some trouble accessing the AWS credentials in the SDK Store, but it seems to only be a problem when running under IIS. If I hit the same code by invoking an NUnit test with ReSharper the dependency injection works and the S3 client is able to authenticate. IAmazonS3 s3Client = new AmazonS3Client(); Has anyone else run into this problem? How were you able to get the dependency injection to work? [Edit] The credential file approach has been recommended for use with IIS because the

Multiple S3 buckets in the same CloudFront distribution

懵懂的女人 提交于 2021-02-07 12:28:28
问题 I created a Cloudfront distribution with a CNAME images.domain.com with SSL, and I have 2 S3 buckets: one for user uploads, one for product pictures The default bucket is the uploads bucket I would like to use the same CloudFront for both buckets So I added the 2 buckets as origins and created a "Behavior", with the path /products/* using my product bucket as origin My "Behaviors" are : /products/* to: products bucket (precedence = 0) Default (*) to: uploads bucket (precedence = 1) When I go