amazon-s3

CreateMultipartUpload operation - AWS policy items needed?

故事扮演 提交于 2020-01-03 15:34:12
问题 I'm doing multipart upload via aws cli console but getting this error; A client error (AccessDenied) occurred when calling the CreateMultipartUpload operation: Access Denied Below is my policy, am I missing something in there? Thanks. { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "s3:ListAllMyBuckets" ], "Resource": "arn:aws:s3:::*" }, { "Effect": "Allow", "Action": [ "s3:ListBucket", "s3:GetBucketLocation" ], "Resource": "arn:aws:s3:::mybucket" }, { "Effect":

Does generating a pre-signed URL to an S3 object using the AWS SDK cost anything?

寵の児 提交于 2020-01-03 15:30:35
问题 Using the AWS SDK in any language, you can generate a pre-signed URL to an PRIVATE S3 object and then anyone who has the URL can use it download the object. This is explained here: https://docs.aws.amazon.com/AmazonS3/latest/dev/ShareObjectPreSignedURL.html This is a great way to reduce load on your own server. You can pass off the actual download work to S3 if your clients with follow a redirect. My question is, does actually generating that URL cost anything--I mean actual MONEY. I

Facing image loading issue from AWS S3

做~自己de王妃 提交于 2020-01-03 14:33:02
问题 I have API in php and in that API I have upload images in AWS S3 is working as expected, however fetch data from S3 it takes too much time to load on a mobile device. Can any one help me to fixed slow image load issue? I'm looking best way to load fast image in android and IOS. Note: I'm not using any thumb for image. 回答1: As you have mentioned image on mobile is taking time as this can be due to different sizes for different screen sizes. Ideally you should look forward for following options

Facing image loading issue from AWS S3

天涯浪子 提交于 2020-01-03 14:29:09
问题 I have API in php and in that API I have upload images in AWS S3 is working as expected, however fetch data from S3 it takes too much time to load on a mobile device. Can any one help me to fixed slow image load issue? I'm looking best way to load fast image in android and IOS. Note: I'm not using any thumb for image. 回答1: As you have mentioned image on mobile is taking time as this can be due to different sizes for different screen sizes. Ideally you should look forward for following options

Access Denied error with Amazon S3

↘锁芯ラ 提交于 2020-01-03 11:46:34
问题 I'm getting an Access Denied error with Amazon S3 and can't figure out why. My settings are as follows: STATIC_URL = 'http://s3.amazonaws.com/%s/' % AWS_STORAGE_BUCKET_NAME What would cause an access denied error? I have verified that my keys are correct. 回答1: The URL you show above would resolve to a bucket within S3. In order to access that bucket successfully with such a URL, the permissions on the bucket would have to grant 'public-read' access to the bucket. In addition, each object or

How can I copy clone/duplicate a folder on S3?

吃可爱长大的小学妹 提交于 2020-01-03 09:11:50
问题 I want to make a copy of the folders and images on my s3 bucket for my development server. How can I do that? 回答1: I just wanted to write an updated answer here: You can now use Amazon's AWS Management Console (under the S3 tab) to right click on any folder (or file) in Bucket A, click Copy , then navigate to Bucket B and right click and click Paste This makes it extremely easy to copy the contents of your production bucket over to your dev bucket. 回答2: If you are using linux, and just want

Unable to verify certificate - please set 'ENV['SSL_CERT_FILE'] = path_to_file'

余生长醉 提交于 2020-01-03 08:24:27
问题 I have a Rails 4 application that uses Carrierwave to upload files. After installing Fog to upload files to Amazon 3, I started getting the following error when uploading files: Excon::Errors::SocketError in VideosController#create Unable to verify certificate, please set `Excon.defaults[:ssl_ca_path] = path_to_certs`, `ENV['SSL_CERT_DIR'] = path_to_certs`, `Excon.defaults[:ssl_ca_file] = path_to_file`, `ENV['SSL_CERT_FILE'] = path_to_file`, `Excon.defaults[:ssl_verify_callback] = callback`

Reading the data written to s3 by Amazon Kinesis Firehose stream

拜拜、爱过 提交于 2020-01-03 06:57:39
问题 I am writing record to Kinesis Firehose stream that is eventually written to a S3 file by Amazon Kinesis Firehose. My record object looks like ItemPurchase { String personId, String itemId } The data is written to S3 looks like: {"personId":"p-111","itemId":"i-111"}{"personId":"p-222","itemId":"i-222"}{"personId":"p-333","itemId":"i-333"} NO COMMA SEPERATION. NO STARTING BRACKET as in a Json Array [ NO ENDING BRACKET as in a Json Array ] I want to read this data get a list of ItemPurchase

Can't Delete Empty S3 Bucket

♀尐吖头ヾ 提交于 2020-01-03 06:45:10
问题 I have an S3 bucket that is 100% empty. Versioning was never enabled on the bucket. However, I still cannot remove the bucket. I have tried via the Console and the CLI tool. On the console it just says "Error" with no error message. From the cli and api it tells me: "An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: The bucket you tried to delete is not empty". I have tried all of the following: aws s3 rb s3://<bucket_name> --force -> BucketNotEmpty aws s3 rm s3://

Filtering nested JSON in AWS Glue

依然范特西╮ 提交于 2020-01-03 06:03:08
问题 We would like to use an AWS-Glue Job to filter JSON messages within an s3 bucket. Here is some example JSON: { "property": {"subproperty1": "A", "subproperty2": "B" }} { "property": {"subproperty1": "C", "subproperty2": "D" }} We want to filter on subproperty1 in ["A", "B"] . This is what we try: applyFilter1 = Filter.apply( frame = datasource0, f = lambda x: x["property.subproperty1"] in ["A", "B"] ) Output is then written so a new s3 bucket as follows: datasink2 = glueContext.write_dynamic