Where are the garbage value stored, and for what purpose?
C was designed to be a relatively low-level language so that it could be used for writing, well, low-level stuff like operating systems. (in fact, it was designed so that UNIX could be written in C) You can simply think of it as assembly code with readable syntax and higher-level constructs. For this reason, C (minus optimizations) does exactly what you ask it to do, nothing more, nothing less.
When you write int x;, the compiler simply allocates memory for the integer. You never asked it to store anything there, so whatever was in that location when your program started stays as such. Most often, it turns out that the pre-existing value is "garbage".
Sometimes, an external program (for eg. a device driver) may write into some of your variables, so it is unnecessary to add another instruction to initialize such variables.