why does this code throw an OutOfMemoryException on i = 690864192?
StringBuilder sb = new StringBuilder();
for (int i = 0; i < Int32.MaxValue;
Each time the StringBuilder allocates a new char[] (by pointing to a previous chunk instance.. which is another StringBuilder).. you're putting an insane amount of pressure on the garbage collector.
There are memory restrictions on the size of objects. Your object will likely max out your assigned memory for your applications process before reaching the end... hence OutOfMemoryException in either case.
The last StringBuilder instance you have points to the last instance before it maxed out.. that one also points to the previous instance before it maxed out... etc. You have a gigantic graph of roots for the GC that never get cleaned up.