Google Cloud Functions - how do I authenticate to AWS S3 bucket?

问题

I am trying to get a Google Cloud Function in Python 3.7 to take a file from Google Cloud Storage and upload it into AWS S3. In the command line, I would authenticate with awscli and then use the gsutil cp command to copy the file across. I have translated this process into python as:

import subprocess
def GCS_to_s3(arg1, arg2):
    subprocess.call(["aws configure set aws_access_key_id AKIA********"], shell=True)
    subprocess.call(["aws configure set aws_secret_access_key EgkjntEFFDVej"], shell=True)
    subprocess.call(["gsutil cp gs://test_bucket/gcs_test.csv s3://mybucket)"], shell=True)`

The requirements.txt is:

awscli google-cloud-storage

This function deploys successfully and runs successfully but the output does not show up in the S3 bucket.

What would be the best way of writing such a function?

回答1:

You'll probably want to use the boto3 Python package instead, since the command-line AWS tools aren't available or installable for Cloud Functions. There's a number of ways to configure credentials as well.

来源：https://stackoverflow.com/questions/52693824/google-cloud-functions-how-do-i-authenticate-to-aws-s3-bucket