amazon-s3

How do I get images uploaded to s3 and get the link url as a response

感情迁移 提交于 2020-05-17 06:27:04
问题 Please I am looking for a real help. I followed the Image s3 upload Node.JS | WYSIWYG Javascript HTML Editor | Froala tutorial to be able to upload images to my S3 bucket but all efforts proved abortive. I could upload the images locally on my computer, and get the link url to display it right in the editor. I can't do this to S3. I am using express node.js and pug template engine. The following is my froala script file that initiate the s3Hash and the editor $.get('/s3/posts-photos' , {})

Disable Cloudfront cache if file is not found

六眼飞鱼酱① 提交于 2020-05-17 06:24:26
问题 I created a Cloudfront distribution in front of an S3 bucket with a RoutingRule to redirect to a lambda function if the requested file is not found. I´m using this to resize images. Desired flow: Request the file to Cloudfront File not found in Cloudfront check S3 File not found in S3 redirect to the lambda function Lambda will find the original file, resize it and redirect back to the Cloudfront url. Redirect rule set on s3 website: <RoutingRules> <RoutingRule> <Condition> <KeyPrefixEquals/>

AWS Athena null values are replaced by N after table is created. How to keep them as it is?

大憨熊 提交于 2020-05-17 06:22:05
问题 I'm creating a table in Athena from csv data in S3. The data has some columns quoted, so I use: ROW FORMAT SERDE 'org.apache.hadoop.hive.serde2.OpenCSVSerde' WITH SERDEPROPERTIES ( "separatorChar" = ",", 'serialization.null.format' = '') The serde works fine but then the null values in the resultant table are replaced with N. How can I keep the null values as empty or like Null etc, but not as N. Thanks. 来源: https://stackoverflow.com/questions/61020631/aws-athena-null-values-are-replaced-by-n

Does AWS S3 GetObject provide random access?

时间秒杀一切 提交于 2020-05-17 05:45:06
问题 I can provide HTTP Range headers to AWS S3's GetObject to request a specified range of bytes of an object. Is it truly random access, or does S3 have to process all of the object before that range before returning my requested range? Is the range header simply reducing the bytes transferred, or does it also provide efficient random access? 回答1: I did a quick test with a 2GB file and executed ranged gets for 8 bytes at various offsets in the file (including start, middle, and end). The total

How to use asyncio to download files on s3 bucket

ぐ巨炮叔叔 提交于 2020-05-16 12:57:48
问题 I'm using the following code to download all my files in a s3 bucket: def main(bucket_name, destination_dir): bucket = boto3.resource('s3').Bucket(bucket_name) for obj in bucket.objects.all(): if obj.key.endswith('/'): continue destination = '%s/%s' % (bucket_name, obj.key) if not os.path.exists(destination): os.makedirs(os.path.dirname(destination), exist_ok=True) bucket.download_file(obj.key, destination) I would like to know how to make this asynchronous, if possible. Thank u in advance.

How to use asyncio to download files on s3 bucket

若如初见. 提交于 2020-05-16 12:57:42
问题 I'm using the following code to download all my files in a s3 bucket: def main(bucket_name, destination_dir): bucket = boto3.resource('s3').Bucket(bucket_name) for obj in bucket.objects.all(): if obj.key.endswith('/'): continue destination = '%s/%s' % (bucket_name, obj.key) if not os.path.exists(destination): os.makedirs(os.path.dirname(destination), exist_ok=True) bucket.download_file(obj.key, destination) I would like to know how to make this asynchronous, if possible. Thank u in advance.

Django + AWS S3 Bucket: Authenticated Access to S3 Bucket

荒凉一梦 提交于 2020-05-16 07:05:12
问题 I have a S3 bucket with a special directory that I'm dumping user uploaded images that I want to restrict to authenticated users. It shouldn't be restricted to just the user that uploaded the image. I'm really trying to avoid setting up a db model for this. Is there anyway to block anonymous members from the user uploaded images? So, I have two questions: How do I go about setting up an anonymous-user-blocked S3 bucket? AND How should I access this bucket within a is_authenticated conditional

Django + AWS S3 Bucket: Authenticated Access to S3 Bucket

隐身守侯 提交于 2020-05-16 07:05:07
问题 I have a S3 bucket with a special directory that I'm dumping user uploaded images that I want to restrict to authenticated users. It shouldn't be restricted to just the user that uploaded the image. I'm really trying to avoid setting up a db model for this. Is there anyway to block anonymous members from the user uploaded images? So, I have two questions: How do I go about setting up an anonymous-user-blocked S3 bucket? AND How should I access this bucket within a is_authenticated conditional

S3 Server access logging vs CloudTrail logs

匆匆过客 提交于 2020-05-16 03:42:28
问题 Given that both services are enabled (A single S3 bucket with Server Access Logging enabled and CloudTrail with object-level logging enabled for that bucket): 1. What events will initiate logging from both services? 2. In such a case, what data will one service contain that the other will not? 3. What events will result in a log created by only one of the services? I am having a hard time understanding the logical difference between those two, as both support object level logging. 回答1: This

Apache Ozone + AWS S3 .Net API: PutObject is creating a bucket instead of a key

拜拜、爱过 提交于 2020-05-16 02:27:51
问题 I am trying to create keys in apache OZone using AWS S3 API for .NET. The key I am trying to create must be inside a bucket called "test" that I created using AWS S3 CLI. My code: static async Task WriteFile() { AmazonS3Config config = new AmazonS3Config(); config.ServiceURL = "http://myApacheOzoneEndpoint:8744"; // This port is mapped from a docker container to (not the original endpoint port for Ozone) AWSCredentials credentials = new BasicAWSCredentials("testuser/scm@EXAMPLE.COM",