It is known that if you read data from disc you are IO bound and you can process/parse the read data much faster than you can read it from disc.
But this common wis
IF you work with big files the fastest you can get is AFAIK the MemoryMappedFile class (new in .NET 4).
By this you basically use the OS FileSystem cache-manager memory pages directly... if this is not fast enough then you would need to write your own filesystem...
As for the GC the behaviour can be customized via app.config - for example:
For the GC customization options see http://msdn.microsoft.com/en-us/library/6bs4szyc.aspx - esp. / .