Thread not run immediately when using more than 4 BackgroundWorker

Question

I use multiple BackgroundWorker control to run some task as multithreading. But I found that when using more than 4 BackgroundWoker, the one from 4th forward delay more than second to actually execute from when calling RunWorkerAsync.

Could help me how can I start all backgroundworker immediately?

class TaskLog
{
    public int task_id;
    public DateTime call_time;
    public DateTime start_time;
    public DateTime end_time;
}

BackgroundWorker[] bws = new BackgroundWorker[18];
int[] tasks = new int[] { 1, 2, 3, 4, 5, 6, 7, 8, 9, 10 };
Queue<TaskLog> queueTask;
TaskLog[] records;
int task_complete = 999;

private void button1_Click(object sender, EventArgs e)
{
    if (task_complete < tasks.Length) return;

    task_complete = 0;

    records = tasks.Select(t => new TaskLog { task_id = t }).ToArray();
    queueTask = new Queue<TaskLog>(records);

    for (int i = 0; i < bws.Length && queueTask.Count > 0; ++i)
    {
        bws[i] = new BackgroundWorker();
        bws[i].DoWork += new DoWorkEventHandler(download_vid_work);
        bws[i].RunWorkerCompleted += new RunWorkerCompletedEventHandler(download_vid_complete);

        var x = queueTask.Dequeue();
        x.call_time = DateTime.Now;

        bws[i].RunWorkerAsync(x);
        //Debug.WriteLine("start " + x.task_id);
    }
}

void download_vid_work(object sender, DoWorkEventArgs e)
{
    var record = (TaskLog)e.Argument;
    record.start_time = DateTime.Now;
    //Debug.WriteLine("actually start " + record.task_id);

    Thread.Sleep(10000); // 10s
    e.Result = record;
}

void download_vid_complete(object sender, RunWorkerCompletedEventArgs e)
{
    var record = (TaskLog)e.Result;
    record.end_time = DateTime.Now;
    //Debug.WriteLine("complete " + item.ToString());

    ++task_complete;
    if (task_complete == tasks.Length)
    {
        Debug.WriteLine("all tasks are completed!");
        foreach (var r in records)
        {
            Debug.WriteLine("task {0} delay time: {1}", r.task_id, (r.start_time - r.call_time).TotalMilliseconds.ToString("0,0"));
        }
    }
    else if (queueTask.Count > 0)
    {
        var bw = (BackgroundWorker)sender;
        var nextTask = queueTask.Dequeue();
        bw.RunWorkerAsync(nextTask);
        nextTask.call_time = DateTime.Now;
    }
}

Here is log result after run:

all tasks are completed!
task 1 delay time: 22
task 2 delay time: 24
task 3 delay time: 24
task 4 delay time: 23
task 5 delay time: 1,005
task 6 delay time: 2,002
task 7 delay time: 3,003
task 8 delay time: 4,003
task 9 delay time: 5,004
task 10 delay time: 6,005

Answer 1

The ThreadPool class, which manages the thread pool threads used for BackgroundWorker (and other needs), does not maintain an infinite number of worker threads ready to run.

You can configure the actual number of idle threads (*), using the ThreadPool.SetMinThreads() method. As you can see in your case, when you initially start your program, there are four idle threads ready to accept work right away. The default number of idles threads depends on a variety of things related to the OS version and configuration.

Once there are more queued work items for the thread pool than there are threads to service them, the ThreadPool class does not create new threads right away. It waits for a short period of time (as you can see from your test, one second), on the assumption that it's possible one of the other tasks may finish soon and it will be able to reuse that thread rather than going to all the trouble of creating yet another thread (which incurs its own overhead and would even slow down the work of the threads already running).

In general, you should avoid overriding the default values for the thread pool, as they are generally set correctly given your OS version, hardware, etc. For example, it won't help to have more CPU-bound threads running than you have CPU cores on the machine. Letting the ThreadPool class decide when and how to run your worker threads is usually the best approach.

(*) The above is a bit of an over-simplification. In newer versions of .NET, the minimum number of threads may or may not actually exist at any given time. If work items are queued when there are fewer than the minimum number, ThreadPool will immediately create new threads as needed up to the minimum. Beyond that, it then shifts to its more elaborate creation and scheduling logic.