amazon-s3

How to use the default AWS credentials chain for an S3 backed Maven repository in a Gradle build?

安稳与你 提交于 2020-03-18 04:54:23
问题 According to Gradle documention (Example 50.27), we can use S3 backed Maven repositories with Gradle 2.4. However, the only example given in the docs passes explicit AWS credentials to the S3 repository declaration: repositories { maven { url "s3://someS3Bucket/maven2" credentials(AwsCredentials) { accessKey "someKey" secretKey "someSecret" } } } I need to do the same thing, but I want Gradle to use the DefaultAWSCredentialsProviderChain, from the AWS JDK, instead of explicit credentials.

script to download file from Amazon S3 bucket

…衆ロ難τιáo~ 提交于 2020-03-18 04:18:11
问题 Trying to write script to download file from Amazon S3 bucket. Having trouble with the example on the cURL site. The script below produces: The request signature we calculated does not match the signature you provided. Check your key and signing method. Appreciate any help. #!/bin/sh file="filename.php" bucket="my-bucket" resource="/${bucket}/${file}" contentType="text/html" dateValue="`date +'%a, %d %b %Y %H:%M:%S %z'`" stringToSign="GET\n\n${contentType}\n${dateValue}\n${resource}" s3Key=

How to delete multiple files in S3 bucket with AWS CLI

妖精的绣舞 提交于 2020-03-17 05:55:22
问题 Suppose I have an S3 bucket named x.y.z In this bucket, I have hundreds of files. But I only want to delete 2 files named purple.gif and worksheet.xlsx Can I do this from the AWS command line tool with a single call to rm ? This did not work: $ aws s3 rm s3://x.y.z/worksheet.xlsx s3://x.y.z/purple.gif Unknown options: s3://x.y.z/purple.gif From the manual, it doesn't seem like you can delete a list of files explicitly by name. Does anyone know a way to do it? I prefer not using the -

How to delete multiple files in S3 bucket with AWS CLI

心已入冬 提交于 2020-03-17 05:55:05
问题 Suppose I have an S3 bucket named x.y.z In this bucket, I have hundreds of files. But I only want to delete 2 files named purple.gif and worksheet.xlsx Can I do this from the AWS command line tool with a single call to rm ? This did not work: $ aws s3 rm s3://x.y.z/worksheet.xlsx s3://x.y.z/purple.gif Unknown options: s3://x.y.z/purple.gif From the manual, it doesn't seem like you can delete a list of files explicitly by name. Does anyone know a way to do it? I prefer not using the -

PutObject into directory Amazon s3 / PHP

故事扮演 提交于 2020-03-13 07:10:08
问题 I need to upload my files inside specific directories that I created on my amazon s3 storage. I always uploaded the files on the "absolute path" of my bucket doing something like so: $s3->putObject(array( 'Bucket' => $bucket, 'ContentType' => $mime, 'Key' => $localImage, 'ACL' => 'public-read', 'SourceFile' => $localImage, 'CacheControl' => 'max-age=172800', "Expires" => gmdate("D, d M Y H:i:s T", strtotime("+5 years")), 'Metadata' => array( 'profile' => $localImage, ), )); How can I define

PutObject into directory Amazon s3 / PHP

巧了我就是萌 提交于 2020-03-13 07:10:06
问题 I need to upload my files inside specific directories that I created on my amazon s3 storage. I always uploaded the files on the "absolute path" of my bucket doing something like so: $s3->putObject(array( 'Bucket' => $bucket, 'ContentType' => $mime, 'Key' => $localImage, 'ACL' => 'public-read', 'SourceFile' => $localImage, 'CacheControl' => 'max-age=172800', "Expires" => gmdate("D, d M Y H:i:s T", strtotime("+5 years")), 'Metadata' => array( 'profile' => $localImage, ), )); How can I define

Amazon S3 Creating Folder through .NET SDK vs through Management Console

泪湿孤枕 提交于 2020-03-13 06:12:07
问题 I'm trying to determine if a folder exists on my Amazon S3 Bucket and if it doesn't I want to create it. At the moment I can create the folder using the .NET SDK as follows: public void CreateFolder(string bucketName, string folderName) { var folderKey = folderName + "/"; //end the folder name with "/" var request = new PutObjectRequest(); request.WithBucketName(bucketName); request.StorageClass = S3StorageClass.Standard; request.ServerSideEncryptionMethod = ServerSideEncryptionMethod.None; /

Amazon S3 REST API 403 error c#

淺唱寂寞╮ 提交于 2020-03-13 05:17:30
问题 I'm trying to get some code working to fetch a file from S3 using the REST API via C#. I've seen other people doing similar things but for some reason I keep getting a 403 error. I've tried to do the same thing with the AWS SDK for .Net and it works so I assume it's the way I'm creating the authorization header. Is anyone able to shed any light on this please? string awsAccessId = "***"; string awsSecretKey = "***"; string bucketName = "thebucket"; string httpDate = DateTime.Now.ToString("ddd

How to avoid small file problem while writing to hdfs & s3 from spark-sql-streaming

。_饼干妹妹 提交于 2020-03-08 09:14:46
问题 Me using spark-sql-2.3.1v , kafka with java8 in my project. With --driver-memory 4g \ --driver-cores 2 \ --num-executors 120 \ --executor-cores 1 \ --executor-memory 768m \ At consumer side , me trying to write the files in hdfs Me using something like this below code dataSet.writeStream() .format("parquet") .option("path", parqetFileName) .option("mergeSchema", true) .outputMode("Append") .partitionBy("company_id","date") .option("checkpointLocation", checkPtLocation) .trigger(Trigger

Fallback to S3 website subfolder for Angular SPA

旧街凉风 提交于 2020-03-06 10:58:30
问题 I have recently been following the two questions to create a website that serves a multitude of static sites from a S3 domain subfolder: Serving a multitude of static sites from a wildcard domain in AWS Lambda script to direct to fallback S3 domain subfolder when not found What I failed to realise is that within each sub-domain there is a compiled Angular application present. Angular has SPAs which need to fall back to index.html mto effectively let the SPA handle routing. My current