There's plenty of answers above for you to look at, but I thought I'd add these resources that'll give you a flavour of what happens. Basically, on Windows and linux, someone has tried to create the tiniest executable possible; in Linux, ELF, windows, PE.
- Tiny PE: http://www.phreedom.org/solar/code/tinype/
- Small ELF files: http://www.muppetlabs.com/~breadbox/software/tiny/teensy.html
Both run through what is removed and why and you use assemblers to construct ELF files without using the -felf like options that do it for you.
Hope that helps.
Edit - you could also take a look at the assembly for a bootloader like the one in truecrypt http://www.truecrypt.org or "stage1" of grub (the bit that actually gets written to the MDR).
