Out of Memory Exception When Using HttpWebRequest to Stream Large File

问题

I get an Out of Memory Exception when using Http.Put of a large file. I am using an asynchronous model as shown in the code. I am trying to send 8K blocks of data to a Windows 2008 R2 server. The exception consistently occurs when I attempt to write a block of data that exceeds 536,868,864 bytes. The exception occurs on the requestStream.Write method in the code snippet below.

Looking for reasons why?

Note: Smaller files are PUT OK. Logic also works if I write to a local FileStream. Running VS 2010, .Net 4.0 on Win 7 Ultimate client computer.

   HttpWebRequest request = (HttpWebRequest)WebRequest.Create("Http://website/FileServer/filename");
   request.Method = WebRequestMethods.Http.Put;
   request.SendChunked = true;
   request.AllowWriteStreamBuffering = true;
   ...

   request.BeginGetRequestStream( new AsyncCallback(EndGetStreamCallback), state);
   ...

   int chunk = 8192; // other values give same result
   ....

   private static void EndGetStreamCallback(IAsyncResult ar) {
        long limit = 0;
        long fileLength;
        HttpState state = (HttpState)ar.AsyncState;

        Stream requestStream = null;
        // End the asynchronous call to get the request stream.

        try {
            requestStream = state.Request.EndGetRequestStream(ar);
            // Copy the file contents to the request stream.

            FileStream stream = new FileStream(state.FileName, FileMode.Open, FileAccess.Read, FileShare.None, chunk, FileOptions.SequentialScan);

            BinaryReader binReader = new BinaryReader(stream);
            fileLength = stream.Length;

            // Set Position to the beginning of the stream.
            binReader.BaseStream.Position = 0;

            byte[] fileContents = new byte[chunk];

            // Read File from Buffer 
            while (limit < fileLength)
            {
                fileContents = binReader.ReadBytes(chunk);

                // the next 2 lines attempt to write to network and server
                requestStream.Write(fileContents, 0, chunk);   // causes Out of memory after 536,868,864 bytes
                requestStream.Flush();  // I get same result with or without Flush

                limit += chunk;
            }

            // IMPORTANT: Close the request stream before sending the request.
            stream.Close();

            requestStream.Close();
        }
    }

回答1:

You apparently have this documented problem. When AllowWriteStreamBuffering is true, it buffers all the data written to the request! So, the "solution" is to set that property to false:

To work around this issue, set the HttpWebRequest.AllowWriteStreamBuffering property to false.

来源：https://stackoverflow.com/questions/6247040/out-of-memory-exception-when-using-httpwebrequest-to-stream-large-file

标签

streaming

out-of-memory