Apparently one must explicitly specify (whitelist) the IP addresses that will be allowed access to a SQL Azure DB. I want, though, to have N-gazillion* users that will acces
It would be advisable to have some sort of middle ware access the db and not your clients directly.
However if you want any IP to be able to connect to the db just add this entry to the firewall list:
Azure Portal -> Databases -> Servers -> Configure and add the following rule: