Speed up loop using multithreading in C# (Question)

Question

Imagine I have an function which goes through one million/billion strings and checks smth in them.

f.ex:

foreach (String item in ListOfStrings)
{
            


        

Answer 1

Not that I have any good articles here right now, but what you want to do is something along Producer-Consumer with a Threadpool.

The Producers loops through and creates tasks (which in this case could be to just queue up the items in a List or Stack). The Consumers are, say, five threads that reads one item off the stack, consumes it by calculating it, and then stores it else where.

This way the multithreading is limited to just those five threads, and they will all have work to do up until the stack is empty.

Things to think about:

• Put protection on the input and output list, such as a mutex.
• If the order is important, make sure that the output order is maintained. One example could be to store them in a SortedList or something like that.
• Make sure that the CalculateSmth is thread safe, that it doesn't use any global state.