Reading an JSON file from S3 using Python boto3

前端 未结 5 940
灰色年华
灰色年华 2020-12-08 04:41

I kept following JSON in S3 bucket \'test\'

{
  \'Details\' : \"Something\" 
}

I am using following code to read this JSON and printing the

5条回答
  •  醉话见心
    2020-12-08 04:58

    I was stuck for a bit as the decoding didn't work for me (s3 objects are gzipped).

    Found this discussion which helped me: Python gzip: is there a way to decompress from a string?

    import boto3
    import zlib
    
    key = event["Records"][0]["s3"]["object"]["key"]
    bucket_name = event["Records"][0]["s3"]["bucket"]["name"]
    
    s3_object = S3_RESOURCE.Object(bucket_name, key).get()['Body'].read()
    
    jsonData = zlib.decompress(s3_object, 16+zlib.MAX_WBITS)
    

    If youprint jsonData, you'll see your desired JSON file! If you are running test in AWS itself, be sure to check CloudWatch logs as in lambda it wont output full JSON file if its too long.

提交回复
热议问题