How can I bypass the 10MB limit of AWS API gateway and POST large files to AWS lambda?

Question

what I want

An API which takes file and some parameters using POST and gives back a JSON response.

curl -X POST          


        

Answer 1

Have an endpoint on your API gateway that generates pre-signed URLs for uploading files to S3. Then your client application can invoke that endpoint to get the pre-signed URL, after which it can upload the file to S3. Then you can have a Lambda function that is triggered by new objects in your S3 bucket that will read the file from S3 and process it.

To post extra parameters along with the file you have a few options:

• Option 1: Post the extra parameters along with the initial request for the pre-signed URL. Have the Lambda function that generates the pre-signed URL store those parameters somewhere like DynamoDB, along with the S3 object key the pre-signed URL is being generated for. Then when the other Lambda function is triggered by the new object appearing in S3 it can look up those extra parameters from DynamoDB.

• Option 2: When uploading the file to S3 via the pre-signed URL, your application can add extra header fields to the upload that will be stored on the S3 object as metadata.