I followed the tutorial below.
https://developers.google.com/eclipse/docs/running_and_debugging_2_0
which basically added a GAE backend to my existing app.
Another possible cause for this problem is not setting the correct applicationId in appengine-web.xml:
APPLICATION_ID_HERE
...