Bulk Read Images from URL using TPL (Task.WhenAll) goes into deadlock when WebClient gives timeout exception

こ雲淡風輕ζ 提交于 2019-12-13 12:45:16

问题


I created a .net core 2.0 console application where I am reading image data from URLs and process them. As images are in bulk, i am doing in threads. Everything works fine but when images does not return any data and throws timeout exception, threads get stuck/hang and does continue. It seems some deadlock but i don't know where exactly.

Below is my code.

int batchSize = 25;
int numberOfBatches = xmlProperties.Count() < batchSize ? 1 : ((((xmlProperties.Count() / batchSize) % 1) == 0) ? (xmlProperties.Count() / batchSize)
                                                                    : (xmlProperties.Count() / batchSize) + 1);
for (int i = 0; i < numberOfBatches; i++)
{
    Task[] Requests = xmlProperties
                        .Skip(i * batchSize)
                        .Take(batchSize)
                        .Select(l => UploadImage(l.PropertyRefNo, l.Images))
                        .ToArray();
    await Task.WhenAll(Requests); // does not return when UploadImage throws exception in WebClient request.
}

public async Task UploadImage(string PropertyRefNo, List<string> Images)
{
    Images = Images.OrderBy(x => x).Take(10).ToList();          

    var index1 = 0;
    foreach (var item in Images)
    {
        try
        {
            index1 = index1 + 1;
            using (MemoryStream imageStream = await ImageUtilities.GetImageData(item)) // Exception comes here in reading data from URL
            {
                // Do Rest Work
            }
        }
        catch (Exception ex)
        {
            await AddLog(0, JobStatus.ImageFailedFull.ToString(), ex.ToString(), PropertyRefNo + " : " + item, 0);
        }
    }
}

public static async Task<MemoryStream> GetImageData(string filename)
{
    try
    {
        using (HttpClient client = new HttpClient())
        {
            var data = await client.GetAsync(filename);
            byte[] imageStream = await data.Content.ReadAsByteArrayAsync();
            return new MemoryStream(imageStream);
        }
    }
    catch (Exception ex)
    {
        throw ex;
    }
}

public class WebClientWithTimeout : WebClient
{
    protected override WebRequest GetWebRequest(Uri address)
    {
        WebRequest wr = base.GetWebRequest(address);
        wr.Timeout = 100000; // 100 seconds
        return wr;
    }
}

Any help would be highly appreciated.

来源:https://stackoverflow.com/questions/54360300/bulk-read-images-from-url-using-tpl-task-whenall-goes-into-deadlock-when-webcl

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!