How to know if all the Thread Pool's thread are already done with its tasks?

Question

I have this application that will recurse all folders in a given directory and look for PDF. If a PDF file is found, the application will count its pages using ITextSharp. I did this by using a thread to recursively scan all the folders for pdf, then if then PDF is found, this will be queued to the thread pool. The code looks like this:

//spawn a thread to handle the processing of pdf on each folder.
                var th = new Thread(() =>
                {
                    pdfDirectories = Directory.GetDirectories(pdfPath);
                    processDir(pdfDirectories);
                });
                th.Start();



 private void processDir(string[] dirs)
        {
            foreach (var dir in dirs)
            {
                pdfFiles = Directory.GetFiles(dir, "*.pdf");
                processFiles(pdfFiles);

                string[] newdir = Directory.GetDirectories(dir);
                processDir(newdir);
            }
        }



private void processFiles(string[] files)
        {
            foreach (var pdf in files)
            {
                ThreadPoolHelper.QueueUserWorkItem(
                    new { path = pdf },
                    (data) => { processPDF(data.path); }
                    );
            }
        }

My problem is, how do i know that the thread pool's thread has finished processing all the queued items so i can tell the user that the application is done with its intended task?

Answer 1

Generally I would do something like this by having a counter variable. For each work item you queue in the ThreadPool add one to the counter variable. Then when it is processed you would decrease the counter variable.

Be sure that you do the incrementing and decrementing via the methods on the Interlocked class as this will ensure that things are done in a thread-safe manner.

Once the counter hits zero you could flag that the tasks are completed using a ManualResetEvent

If you have access to .NET 4 then you can use the new CountdownEvent class to do a similar thing.

Answer 2

1) How to know if all threads are finished?

You will have to let the Threads do their own check-in/check-out, by bracketing your code between:

Interlocked.Increment(ref jobCounter);
// your code
Interlocked.Decrement(ref jobCounter);

If this messes up your anonymous delegates too much then just use wrapper methods. You are probably going to have to add exception handling too.

The interlocked approach still eaves the problem of waiting for it to become 0, a loop with Sleep() is a weak but in this case viable solution.

2) You are starting threads in a recursive Tree walker. Be careful, you are probably creating too much of them and that will hurt performance.