Is it possible for a self-hosted OWIN Web API to run under a non-administrator account? I have already tried dozens of url reservations and nothing works. The service fail
Your service is running (most likely) under the LocalSystem (SYSTEM) account. This account is not in the Everyone security principal.
In short, to solve this, either make the namespace reservation for Anonymous Logon or change your service to run under the Network Service account which happens to be in the Everyone principal.
Third option is, of course, to create a new local/domain user, create the reservation for it and have the service run under this account. But then you'd have to worry about setting proper security permissions for it, so I'd go with one of the first two options.