AWS: how to fix S3 event replacing space with '+' sign in object key names in json

问题

I have a lamba function to copy objects from bucket 'A' to bucket 'B', and everything was working fine, until and object with name 'New Text Document.txt' was created in bucket 'A', the json that gets built in S3 event, key as "key": "New+Text+Document.txt".

the spaces got replaced with '+'. I know it is a known issue by seraching on web. But I am not sure how to fix this and the incoming json itself has a '+' and '+' can be actually in the name of the file. like 'New+Text Document.txt'.

So I cannot blindly have logic to space '+' by ' ' in my lambda function.

Due to this issue, when code tries to find the file in bucket it fails to find it.

Please suggest.

回答1:

I came across this looking for a solution for a lambda written in python instead of java; "urllib.parse.unquote_plus" worked for me, it properly handled a file with both spaces and + signs:

from urllib.parse import unquote_plus
import boto3


bucket = 'testBucket1234'
# uploaded file with name 'foo + bar.txt' for test, s3 Put event passes following encoded object_key
object_key = 'foo %2B bar.txt'
print(object_key)
object_key = unquote_plus(object_key)
print(object_key)

client = boto3.client('s3')
client.get_object(Bucket=bucket, Key=object_key)

回答2:

What I have done to fix this is

java.net.URLDecoder.decode(b.getS3().getObject().getKey(), "UTF-8")


{
    "Records": [
        {
            "s3": {
                "object": {
                    "key": "New+Text+Document.txt"
                }
            }
        }
    ]
}

So now the JSon value, "New+Text+Document.txt" gets converted to New Text Document.txt, correctly.

This has fixed my issue, please suggest if this is very correct solution. Will there be any corner case that can break my implementation.

回答3:

I think in Java you should use:

getS3().getObject().getUrlDecodedKey()

method that returns decoded key, instead of

getS3().getObject().getKey()

回答4:

Since we are sharing for other runtimes here is how to do it in NodeJS:

const srcKey = decodeURIComponent(event.Records[0].s3.object.key.replace(/\+/g, " "));

From the AWS docs here

回答5:

Agree with Scott. for me create object event was appending %3 for semicolon : i have to replace it twice to get correct s3 url

Python code:

    def lambda_handler(event, context):
    logger.info('Event: %s' % json.dumps(event))
    source_bucket = event['Records'][0]['s3']['bucket']['name']
    key_old = event['Records'][0]['s3']['object']['key']
    key_new = key_old.replace('%3',':')
    key = key_new.replace(':A',':')
    logger.info('key value')
    logger.info(key)

回答6:

in ASP.Net has UrlDecode. The sample is below.

HttpUtility.UrlDecode(s3entity.Object.Key, Encoding.UTF8)

来源：https://stackoverflow.com/questions/44779042/aws-how-to-fix-s3-event-replacing-space-with-sign-in-object-key-names-in-js

标签

java

json

amazon-web-services

amazon-s3

aws-lambda