Uploading HTTP progress tracking

Question

I\'ve got WPF application I\'m writing that posts files to one of social networks. Upload itself working just fine, but I\'d like to provide some indication of how far along

Answer 1

This one has been bugging me for at least one day. I have started with using WebClient.UploadFileAsync, next tried the ProgressMessageHandler for HttpClient then rolled my own HttpContent for the HttpClient API. None of those approaches worked (for me).

It appears HttpWebRequest, which sits at the bottom of most (all?) .NET Http abstraction like WebClient and HttpClient, buffers the request and response stream by default, which I confirmed by looking at it in ILSpy.

As others have noted, you can make your request use chunked encoding one way or another which will effectively disable buffering the request stream, but still this is not going to fix the progress reporting.

I found that it was necessary to flush the request stream after each block that I send in order to accurately reflect sending progress, or else your data will simply be buffered one step further down the pipeline (probably somewhere in NetworkStream or OS, didn't check). The sample code below works for me and also does a minimalistic job at translating back from a HttpWebResponse to HttpResponseMessage (which you may not need, YMMV).

public async Task<HttpResponseMessage> UploadFileAsync( string uploadUrl, string absoluteFilePath, Action<int> progressPercentCallback )
    {
        var length = new FileInfo( absoluteFilePath ).Length;

        var request = new HttpWebRequest( new Uri(uploadUrl) ) {
            Method = "PUT",
            AllowWriteStreamBuffering = false,
            AllowReadStreamBuffering = false,
            ContentLength = length
        };

        const int chunkSize = 4096;
        var buffer = new byte[chunkSize];

        using (var req = await request.GetRequestStreamAsync())
        using (var readStream = File.OpenRead(absoluteFilePath))
        {
            progressPercentCallback(0);
            int read = 0;
            for (int i = 0; i < length; i += read)
            {
                read = await readStream.ReadAsync( buffer, 0, chunkSize );
                await req.WriteAsync( buffer, 0, read );
                await req.FlushAsync(); // flushing is required or else we jump to 100% very fast
                progressPercentCallback((int)(100.0 * i / length));
            }
            progressPercentCallback(100);
        }

        var response = (HttpWebResponse)await request.GetResponseAsync();
        var result = new HttpResponseMessage( response.StatusCode );
        result.Content = new StreamContent( response.GetResponseStream() );

        return result; 
    }

Answer 2

At fast guess, you are running this code on UI thread. You need to run upload stuff on new thread. At that point you have 2 options. 1) You run timer on UI thread and update UI. 2) You update UI using Invoke(because you can't access UI from another thread) calls to update UI.

Answer 3

You could use the WebClient's UploadFile to upload file rather than using writing file as a file stream. In order to track the percentage of the data received and uploaded you can use UploadFileAsyn and subscribe to its events.

In the code bellow I've used UploadFileAsyn to the upload files synchronously, but it need not to be synchronous as far as you don't dispose the instance of the uploader.

class FileUploader : IDisposable
{
    private readonly WebClient _client;
    private readonly Uri _address;
    private readonly string _filePath;
    private bool _uploadCompleted;
    private bool _uploadStarted;
    private bool _status;

    public FileUploader(string address, string filePath)
    {
        _client = new WebClient();
        _address = new Uri(address);
        _filePath = filePath;
        _client.UploadProgressChanged += FileUploadProgressChanged;
        _client.UploadFileCompleted += FileUploadFileCompleted;
    }

    private void FileUploadFileCompleted(object sender, UploadFileCompletedEventArgs e)
    {
        _status = (e.Cancelled || e.Error == null) ? false : true;
        _uploadCompleted = true;
    }

    private void FileUploadProgressChanged(object sender, UploadProgressChangedEventArgs e)
    {
        if(e.ProgressPercentage % 10 == 0)
        {
            //This writes the pecentage data uploaded and downloaded
            Console.WriteLine("Send: {0}, Received: {1}", e.BytesSent, e.BytesReceived);
            //You can have a delegate or a call back to update your UI about the percentage uploaded
            //If you don't have the condition (i.e e.ProgressPercentage % 10 == 0 )for the pecentage of the process 
            //the callback will slow you upload process down
        }
    }

    public bool Upload()
    {

        if (!_uploadStarted)
        {
            _uploadStarted = true;
            _client.UploadFileAsync(_address, _filePath);
        }
        while (!_uploadCompleted)
        {
            Thread.Sleep(1000);
        }
        return _status;
    }

    public void Dispose()
    {
        _client.Dispose();
    }
}

Client Code:

            using (FileUploader uploader = new FileUploader("http://www.google.com", @"C:\test.txt"))
        {
            uploader.Upload();
        }

You can register a custom callback (may be a delegate) on the FileUploadProgressChanged event handler to update your WPF UI.

The upload progress changed event get called more often if your callback for the event does any IO then that'll slowdown the download progress. It's best to have infrequent update e.g. the following code update only evey 10% up.

    private int _percentageDownloaded;

    private void FileUploadProgressChanged(object sender, UploadProgressChangedEventArgs e)
    {
        if (e.ProgressPercentage % 10 == 0 && e.ProgressPercentage > _percentageDownloaded)
        {

            _percentageDownloaded = e.ProgressPercentage;
            //Any callback instead of printline
            Console.WriteLine("Send: {0} Received: {1}", e.BytesSent, e.BytesReceived);
        }
    }

Answer 4

my suggestion is to use new HTTPClient class (available in .NET 4.5). It supports progress.

This article helped me a lot with this: http://www.strathweb.com/2012/06/drag-and-drop-files-to-wpf-application-and-asynchronously-upload-to-asp-net-web-api/

My code for upload file:

    private void HttpSendProgress(object sender, HttpProgressEventArgs e)
    {
        HttpRequestMessage request = sender as HttpRequestMessage;
        Console.WriteLine(e.BytesTransferred);
    }

    private void Window_Loaded_1(object sender, RoutedEventArgs e)
    {
        ProgressMessageHandler progress = new ProgressMessageHandler();
        progress.HttpSendProgress += new EventHandler<HttpProgressEventArgs>(HttpSendProgress);

        HttpRequestMessage message = new HttpRequestMessage();
        StreamContent streamContent = new StreamContent(new FileStream("e:\\somefile.zip", FileMode.Open));

        message.Method = HttpMethod.Put;
        message.Content = streamContent;
        message.RequestUri = new Uri("{Here your link}");

        var client = HttpClientFactory.Create(progress);

        client.SendAsync(message).ContinueWith(task =>
        {
            if (task.Result.IsSuccessStatusCode)
            { 

            }
        });
    }

Answer 5

In the first example I think your progress bar is showing how fast you write into the stream from the file on disk - not the actual upload progress (which is why it all happens to 100% really quickly then the upload chugs on*).

I might be wrong ^^ and have no WPF experience but I have uploaded massive files from Silverlight to WCF and the model used there is (as you do) to break up the file into blocks. Send each block. When you get a response from the server ("block 26 received ok"), update the progress bar as really, you can't (or should not) update the progress bar unless you /know/ that block x made it - and a good way to know that is if the server says it got it.

*I wish I could upload 400Mb in 5 mins. Would take me all day...

Answer 6

it's more than year since this question was posted, but I think my post can be usefull for someone.

I had the same problem with showing progress and it behaved exactly like you described. So i decided to use HttpClient which shows upload progress correctly. Then I've encountered interesting bug - when I had Fiddler launched HttpClient started to show its upload progress in unexpected way like in WebClient/HttpWebRequest above so I thinked maybe that was a problem of why WebClient showed upload progres not correctly (I think I had it launched). So I tried with WebClient again (without fiddler-like apps launched) and all works as it should, upload progress has correct values. I have tested in on several PC with win7 and XP and in all cases progress was showing correctly.

So, I think that such program like Fiddler (probably not only a fiddler) has some affect on how WebClient and other .net classes shows upload progress.

this discussion approves it:

HttpWebRequest doesn't work except when fiddler is running