I suggest you get yourself a small microcontroller system to play around with. TI has a small system-on-a-usbstick based on the MSP430 for $20.
You do not need a compiler or an OS to run code on this system. In fact, with only 128 bytes of ram it would be a challenge to get an OS onto it.
If you really want to start from scratch, you the write down your program in assembly code on a piece of paper and translate it by hand to machine code. Then you type in the machine code in a hex editor to get your ones and zeroes. This is really taking it to the extreme and you will not learn anything from doing the machine code by hand. Writing a program in assembly code will teach you a lot, though.
You do need a tool to get your program into flash memory for the cpu to be able to execute it, but once the flash has been programmed, there are no tools, os or anything like that involved.
