First of all there is no one doctype you should be using, but most designers try to make it work within XHTML 1.0 Strict.
A doctype is nothing more than a declaration of what tags you can use within your html (though the browsers can use more or less than what is defined) You can actually open up the doctype file and start reading (XHTML 1.0 Strict)
If you do not specify a doctype, the browser will try its best to guess but not always hits the correct type.
Quirks mode is just a technique used by browsers to be backwards compatible, a great example of quirks mode is how IE renders boxes