I am using Angular UI Router in my angular app and i have enabled HTML5 mode to remove the # form the URL by using $locationProvider in the config.
If you cannot configure your server, configure your web.xml with the following to handle refresh while the user is on an html5 push path.
404 /index.html