amazon-s3

问题 I am new to logstash. I have some logs stored in AWS S3 and I am able to import them to logstash. My question is: is it possible to use the grok filter to add tags based on the filenames? I try to use: grok { match => {"path" => "%{GREEDYDATA}/%{GREEDYDATA:bitcoin}.err.log"} add_tag => ["bitcoin_err"] } This is not working. I guess the reason is "path" only working with file inputs. Here is the structure of my S3 buckets: my_buckets ----A ----2014-07-02 ----a.log ----b.log ----B ----2014-07

问题 I have a paperclip text file attachment (in Rails). My bucket policy is: { "Version": "2008-10-17", "Id": "Policy123", "Statement": [ { "Sid": "Stmt123", "Effect": "Allow", "Principal": { "AWS": "*" }, "Action": [ "s3:GetObjectVersion", "s3:DeleteObject", "s3:DeleteObjectVersion", "s3:GetObject", "s3:PutObject" ], "Resource": "arn:aws:s3:::my_bucket/*" } ] } I want to restrict access to these actions to only be allowed when the request comes from my website. Is it simply a case of updating

问题 I want to save dataframe to s3 but when I save the file to s3 , it creates empty file with ${folder_name} , in which I want to save the file. Syntax to save the dataframe :- f.write.parquet("s3n://bucket-name/shri/test") It saves the file in test folder but it creates $test under shri . Is there a way I can save it without creating that extra folder? 回答1: I was able to do it by using below code. df.write.parquet("s3a://bucket-name/shri/test.parquet",mode="overwrite") 回答2: As far as I know,

问题 I'm new to EC2. I have read a lot about it, watched many videos and tutorial and pretty much familiar with how everything work. I still have few question that I can't seem to find direct answers to. If I have 3 instances (linux) with load balancing all serving the same site and the site is a dynamic php/mysql driven where users post files forum threads every second, how is the database and files synced to all 3 instances in real time. Do I need to have the database on RDS where every instance

问题 I'm trying to get an built that allows users to upload a file directly to my Amazon S3 bucket, from a NodeJS powered website. It seems the only tutorials out there, other than the actual amazon docs for this are all very out of date. I've been following this tutorial , for the basic info, but again it's out dated. It doesn't have the method calls to crypto correct, as it tries to pass a raw JavaScript object to the update method, which throws an error because it's not a string or buffer. I've

问题 I'm trying to get an built that allows users to upload a file directly to my Amazon S3 bucket, from a NodeJS powered website. It seems the only tutorials out there, other than the actual amazon docs for this are all very out of date. I've been following this tutorial , for the basic info, but again it's out dated. It doesn't have the method calls to crypto correct, as it tries to pass a raw JavaScript object to the update method, which throws an error because it's not a string or buffer. I've

问题 I am trying to export an EC2 instance (instance created with AMI which was imported earlier using VM import export service) to S3 bucket which is constantly failing . I also tried changing the bucket access to public still the error continues . I used both python SDK and CLI commands where both returns the same error CLI used : aws ec2 create-instance-export-task --instance-id i-***** --target-environment vmware --export-to-s3-task file://C:\file.json json file containing bucket and VM image

问题 I hope this question isn't too rudimentary, but I'm confused... In the S3 documentation I read: All HTTP queries have an expiration parameter that allows you to set how long the query will be valid. For example, you can configure a web page graphic to expire after a very long period of time or a software download to only last for 24 hours. For a publicly accessible data object (file), does this mean that the data object (file) itself will not be valid anymore, or that the browser will simply

问题 I need to deploy a React App on AWS S3 using SSL and managing caching. What are the required steps, and what are some of the problems I might encounter? 回答1: Why do this? It can be incredibly fast, "serverless" and very inexpensive. Through the CloudFront global endpoints (edge locations), an application can run very quickly with high reliability. By setting another source origin, CloudFront can act as a reverse proxy to an API, eliminating cross-region (CORS) issues and accelerating API

问题 In the aws-sdk's S3 class, what is the difference between upload() and putObject()? They seem to do the same thing. Why might I prefer one over the other? 回答1: The difference between using AWS SDK upload() and putObject() param is as below: If the reported MD5 upon upload completion does not match, it retries. If the file size is large enough, it uses multipart upload to upload parts in parallel. Retry based on the client's retry settings. You can use for Progress reporting. Sets the