C# System.Diagnostics.Process redirecting Standard Out for large amounts of data

Question

I running an exe from a .NET app and trying to redirect standard out to a streamreader. The problem is that when I do

myprocess.exe >> out.txt

out.txt is cl

Answer 1

Yes, that's about right. There is a buffer that stores the process output, usually between 1 and 4KB in the common CRT implementations. One small detail: that buffer is located in the process you start, not the .NET program.

Nothing very special needs to happen when you redirect to a file, the CRT directly writes it. But if you redirect to your .NET program then output goes from the buffer into a pipe. Which then takes a thread switch to your program so you can empty the pipe. Back and forth a good 700 times.

Yes, not fast. Easily fixed though, call setvbuf() in the program you are running to increase the stdout and stderr output buffer sizes. Then again, that takes having the source code of that program.

Anticipating a problem with that: maybe you ought to use cmd.exe /c to get the redirection to a file, then read the file.