task-parallel-library

Aws Sqs Consumer - Poll only when messages can be processed immediately

假装没事ソ 提交于 2021-01-27 18:19:53
问题 I'm trying to create an AWS SQS windows service consumer that will poll messages in batch of 10. Each messages will be executed in its own task for parallel execution. Message processing includes calling different api's and sending email so it might take some time. My problem is that first, I only want to poll the queue when 10 messages can be processed immediately. This is due to sqs visibility timeout and having the received messages "wait" might go over the visibility timeout and be "back"

Where does async and await end? Confusion

大憨熊 提交于 2021-01-27 06:17:13
问题 I have a program which has no purpose but to help me understand how async and await works. It's a console application which parses XML and waits for a name to be returned, either the surname or the first name. Here is the code: static void Main(string[] args) { Task<string> name = GetFirstParsedName(); name.Wait(); if (name.IsCompleted) { Console.WriteLine(name.Result); } Console.ReadLine(); } static async Task<string> GetFirstParsedName() { string xmlSnippet = @"<person> <FirstName>Chamir<

Where does async and await end? Confusion

拜拜、爱过 提交于 2021-01-27 06:17:13
问题 I have a program which has no purpose but to help me understand how async and await works. It's a console application which parses XML and waits for a name to be returned, either the surname or the first name. Here is the code: static void Main(string[] args) { Task<string> name = GetFirstParsedName(); name.Wait(); if (name.IsCompleted) { Console.WriteLine(name.Result); } Console.ReadLine(); } static async Task<string> GetFirstParsedName() { string xmlSnippet = @"<person> <FirstName>Chamir<

System.Timers.Timer Elapsed intermittently not firing when using an Task.Run with async from Console App

给你一囗甜甜゛ 提交于 2021-01-26 13:23:49
问题 I am using a console application and I have batches of 20 URIs that I need to read from and I have found a massive speed boost by making all tasks and running them in parallel then sorting the results on completion in a different thread (allowing the next batch to be fetched). In the calls I am currently using, each thread blocks when it gets the response stream, I also see there is a async version of the same method GetResponseAsync . I understand there are benefits of freeing up the thread

System.Timers.Timer Elapsed intermittently not firing when using an Task.Run with async from Console App

孤者浪人 提交于 2021-01-26 13:23:19
问题 I am using a console application and I have batches of 20 URIs that I need to read from and I have found a massive speed boost by making all tasks and running them in parallel then sorting the results on completion in a different thread (allowing the next batch to be fetched). In the calls I am currently using, each thread blocks when it gets the response stream, I also see there is a async version of the same method GetResponseAsync . I understand there are benefits of freeing up the thread

System.Timers.Timer Elapsed intermittently not firing when using an Task.Run with async from Console App

大憨熊 提交于 2021-01-26 13:23:07
问题 I am using a console application and I have batches of 20 URIs that I need to read from and I have found a massive speed boost by making all tasks and running them in parallel then sorting the results on completion in a different thread (allowing the next batch to be fetched). In the calls I am currently using, each thread blocks when it gets the response stream, I also see there is a async version of the same method GetResponseAsync . I understand there are benefits of freeing up the thread

System.Timers.Timer Elapsed intermittently not firing when using an Task.Run with async from Console App

早过忘川 提交于 2021-01-26 13:23:05
问题 I am using a console application and I have batches of 20 URIs that I need to read from and I have found a massive speed boost by making all tasks and running them in parallel then sorting the results on completion in a different thread (allowing the next batch to be fetched). In the calls I am currently using, each thread blocks when it gets the response stream, I also see there is a async version of the same method GetResponseAsync . I understand there are benefits of freeing up the thread

TPL DataFlow - Batching on duration or threshold

最后都变了- 提交于 2021-01-22 05:22:51
问题 I have implemented a producer..consumer pattern using TPL data flow. The use case is that code reads messages from the Kafka bus. For efficiency, we need to process messages in batches when going to the database. Is there a way in TPL data flow to hold on to the message and fire whenever a size or duration threshold is hit? Example, the current implementation post the message once it is pulled from the queue. postedSuccessfully = targetBuffer.Post(msg.Value); 回答1: Buffering by count and

TPL DataFlow - Batching on duration or threshold

回眸只為那壹抹淺笑 提交于 2021-01-22 05:22:09
问题 I have implemented a producer..consumer pattern using TPL data flow. The use case is that code reads messages from the Kafka bus. For efficiency, we need to process messages in batches when going to the database. Is there a way in TPL data flow to hold on to the message and fire whenever a size or duration threshold is hit? Example, the current implementation post the message once it is pulled from the queue. postedSuccessfully = targetBuffer.Post(msg.Value); 回答1: Buffering by count and

For a TPL Dataflow: How do I get my hands on all the output produced by a TransformBlock while blocking until all inputs have been processed?

╄→гoц情女王★ 提交于 2021-01-16 04:00:52
问题 I'm submitting a series of select statements (queries - thousands of them) to a single database synchronously and getting back one DataTable per query (Note: This program is such that it has knowledge of the DB schema it is scanning only at run time, hence the use of DataTables ). The program runs on a client machine and connects to DBs on a remote machine. It takes a long time to run so many queries. So, assuming that executing them async or in parallel will speed things up, I'm exploring