Just reading some notes in a purdue lecture about OSs, and it says:
A program sees memory as an array of bytes that goes from address 0 to 2^32-1 (0
Actually, it's not as simple as 2^32 = 4294967296 bytes. You see in x86 protected mode, with paging enabled (that is, what you get when you use any modern OS), you don't address memory locations directly, even though the paging translation mechanism is transparent for client applications.
Of a logical 32 bit memory address, when using 4K pages:
As you can see, you have 2^10 (1024) page directories, in each page directory, you have 2^10 page tables and each page is 2^12 (4096) bytes long, hence 2^32 = 4294967296 bytes. The width of the memory bus is conveniently the same as the word length of the CPU but it's not necessary to be like this at all. In fact, more modern x86 CPUs support PAE which enables addressing more than 4GB (or GiB) even in 32-bit mode.