I have a single page website and would like to achieve the following:
back button working as if it was a normal website
and instead of say,
As mentioned, you will want to use the HTML5 History API. Please note, this API is relatively new and therefore browser support is a concern. At the time of writing, approximately 71% of global Internet users have support for it (see http://caniuse.com/#feat=history for browser support information). Therefore, you will want to ensure you have a fall-back solution for this. You will likely want to use the older #! solution that was popular before the HTML 5 History API was adopted.
If you use the history API to replace, for example, example.com/#!settings
with example.com/settings and a user bookmarks that nicer URL, then when they go to visit it, their browser will make a request to the server for /settings
(which doesn't actually exist in the web server's context). Therefore, you will need to make sure your web server has some redirection rules (i.e. RewriteEngine) such that it can take the pretty URLs and redirect them to the #! version (and then if the user's browser supports the history API it can replace that with the nice URL).
If you aren't very comfortable programming yourself, I'd recommend using a JavaScript library that does a lot of the work for you. I did some quick searching and discovered the following, though there might be better ones out there: https://github.com/browserstate/history.js