AWS API Gateway Proxy Response failure/dropped

Question

Problem:

AWS API Gateway Proxy is not passing back the response from my backend service when using Postman, but works on curl

Description:

I have a backend service that I want to exposed via AWS API gateway. The use of gateway in this case is purely as a HTTP proxy. So,

1. Created a new API
2. Set the Resource as "Proxy" using ANY
3. Gave my backend service address
4. Deployed it (No Auth required, but API Key is needed though)
5. Could call my backend service from the deployment stage url successfully
6. Create a custom domain using a CA cert
7. Created an Alias on my DNS provider
8. 1hr later the Domain is up and working pointing to the cloudfront

Issue

I made a call to the custom domain,

1. API gateway received it
2. Call is logged in the cloudwatch logs, and
3. It is hitting my Backend.
4. Backend replied to API gateway.

But, on my postman, I see a "Could not get any response"

However, If I try this via curl, I am getting the intended response back that was sent by my backend. So, the unclear part is -

1. What is wrong with Postman call or a normal browser url bar based GET to my endpoint?

2. Why is curl successful?

3. and, for my API Gateway endpoint, I have set API Key as mandatory. I have created an API key and assigned it to the API. Interesting part is that, the curl call is still successful even if I do not pass an API Key. Why could API Gateway allow such calls ?

4. Another confusing issue for me is, I have to pass custom values in the Accept, and Content-Type that I use for my API versioning.

Ex: Accept: application/json-v1 is failing but Accept: application/xml-v1 is working.

I think this is something to do with value as API Gateway is not liking it.

I have read through the docs thoroughly and still could not find any reasonable explanation for all these issues. What am I missing that is causing these weird bag of issues ? My google fu so far too was not enough to solve this.

Please help or equally point me to the specific docs in AWS gateway that can help solve this or any discussion threads that I may have missed on these topics.

At least getting an answer for one of these will give me a clarity on my approach.

thanks people!

Answer 1

Quick answer: add header Accept-Encoding:identity

I had the exact same problem when using AWS api-gateway. In my case the solution was to add header Accept-Encoding:identity either from postman or configure the AWS API to send it to my backend by default.

When I call my backend directly from postman I can set the header to gzip and postman will display the response correctly, but when I do it through AWS it seems to be an issue that I haven't been able to address. Setting the mentioned header to Identity will instruct the backend to don't modify the response (or apply the identity function).

Answer 2

1. curl and Postman/browser are both ultimately making HTTP requests to your API Gateway so the task is to figure out what the difference(s) in the requests are. If you run curl with the -v flag you will get verbose output that will include all of the headers sent with the request. Postman/browsers should have something similar, please compare them and you should find a difference. Also to eliminate the custom domain being a possible issue, have you tried making the request to your non-custom domain (*.execute-api) URI?

2. See 1.

3. Make sure your proxy method has the "API Key required" checkbox selected.
4. If you're using mapping template transformations, you'll have to set up a template for each content type you expect to be received from client requests. If you're not using transformations make sure WHEN_NO_TEMPLATE ("When there are no templates defined (recommended)" in the Console UI) passthrough behavior is selected for the integration request. See: 1 and 2

Answer 3

I had the same problem.

The solution that worked for our use case was to disable compression on the Application server. And then enable the Content Encoding feature from AWS API Gateway settings:

Don't forget to redeploy your API Stage from the resources section !!

For your Express.js API Endpoint which is running in AWS Lambda and proxyed through API Gateway, you can modify the app.js to look like this:

   const compression = require('compression');
   if (!process.env.LAMBDA_TASK_ROOT) {
      app(compression()); // compression is only enabled when not in Lambda
   }