I am a little bit confused about the fact that we can just catch an OutOfMemoryException using a try/catch block.
Given the following code:
Not sure whether this answers your question, but a (simplified) explanation on how it decides what objects to clean up is this:
The garbage collector takes every running thread in your program and marks all top-level objects, which means all objects accessible from the stack frames (that is, all the objects pointed by local variables at the points where execution currently is) as well as all objects pointed by static fields.
Then, it marks the next level objects, which means all the objects pointed by all the fields of the previously marked objects. This step is repeated until no new objects are marked.
Because C# doesn't allow pointers in the normal context, once the previous step is completed, it's guaranteed that the non-marked objects are not accessible by subsequent code and therefore can be cleaned up safely.
In your case, if the objects you have allocated to add pressure to the memory manager are not kept by reference, it means that the GC will have the chance to clean them up. Also, keep in mind that OutOfMemoryException refers to the managed memory of your CLR program, while the GC works a little outside of that "box".