Unbuffered Output Very Slow

Question

I generate a very large .csv file from a database using the method outlined in

https://stackoverflow.com/a/13456219/141172

It works fine, up to a point. Whe

Answer 1

I don't know what ASP.NET and IIS are doing exactly with output streaming but maybe too small chunks are being uses. Hook in a BufferedStream with a very big buffer, like 4MB.

According to your comments it worked. Now, tune down the buffer size to save memory and have a smaller working set. Good for cache.

As a subjective comment I'm disappointed that this is even necessary. IIS should use the right buffers automatically which is extremely easy with TCP connections.

EDIT FROM OP

Here is the code derived from this answer

public ActionResult Export()
{
    // Domain specific stuff here
    return new FileGeneratingResult("MyFile.txt", "text/text",
            stream => this.StreamExport(stream), false);
}

private void StreamExport(Stream stream)
{
    using (BufferedStream bs = new BufferedStream(stream, 256*1024))
    using (StreamWriter sw = new StreamWriter(bs))  
    foreach (var stuff in MyData())
    {
        sw.Write(stuff);
    }
}

Answer 2

In Eric's latest update, he mentioned using another thread. I too had this problem for implementing database exports. Here is some example code for the solution I used:

Handling with temporary file stream