I followed the tutorial below.
https://developers.google.com/eclipse/docs/running_and_debugging_2_0
which basically added a GAE backend to my existing app.
I had the same problem and this is how I fixed it.
A little background
I had two versions of the API deployed to App Engine which I believe it should be OK. The admin console didn't show any problem and I was getting the 200 OK code after deploying version 2 of the API in log file.
/_ah/spi/BackendService.getApiConfigs 200 118ms 8kb
but no matter what I tried I was always getting the 404 Not Found error.
I am not sure what the problem was but I think Web Server at App Engine was redirecting all the requests to the version 1. It could be because both versions had the following settings in their web.xml.
SystemServiceServlet
/_ah/spi/*
Solution
I removed version 1 and redeployed version 2. Then everything just worked fine.