Waiting win32 threads

Question

I have a totally thread-safe FIFO structure( TaskList ) to store task classes, multiple number of threads, some of which creates and stores task and the others processes the tasks. TaskList class has a pop_front() method which returns the first task if there is at least one. Otherwise it returns NULL.
Here is an example of processing function:

TaskList tlist;

unsigned _stdcall ThreadFunction(void * qwe)
{
    Task * task;
    while(!WorkIsOver) // a global bool to end all threads.
    {
        while(task = tlist.pop_front())
        {
            // process Task
        }
    }
    return 0;
}

My problem is, sometimes, there is no new task in the task list, so the processing threads enters in an endless loop (while(!WorkIsOver)) and CPU load increases. Somehow I have to make the threads wait until a new task is stored in the list. I think about Suspending and Resuming but then I need extra info about which threads are suspending or running which brings a greater complexity to coding.

Any ideas?

PS. I am using winapi, not Boost or TBB for threading. Because sometimes I have to terminate threads that process for too long, and create new ones immediately. This is critical for me. Please do not suggest any of these two.

Thanks

Answer 1

Assuming you are developing this in DevStudio, you can get the control you want using [IO Completion Ports]. Scary name, for a simple tool.

• First, create an IOCompletion Port: CreateIOCompletionPort
• Create your pool of worker threads using _beginthreadex / CreateThread
• In each worker thread, implement a loop that calls GetQueuedCompletionStatus - The returned lpCompletionKey will be pointing to a work item to process.
• Now, whenever you get a work item to process: call PostQueuedCompletionStatus from any thread - passing in the pointer to your work item as the completion key parameter.

Thats it. 3 API calls and you have implemented a thread pooling mechanism based on a kernel implemented queue object. Each call to PostQueuedCompletionStatus will automatically be deserialized onto a thread pool thread thats blocking on GetQueuedCompletionStatus. The pool of worker threads is created, and maintained - by you - so you can call TerminateThread on any worker threads that are taking too long. Even better - depending on how it is set up the kernel will only wake up as many threads as needed to ensure that each CPU core is running at ~100% load.

NB. TerminateThread is really not an appropriate API to use. Unless you really know what you are doing the threads are going to leak their stacks, none of the memory allocated by code on the thread will be deallocated and so on. TerminateThread is really only useful during process shutdown. There are some articles on the net detailing how to release the known OS resources that are leaked each time TerminateThread is called - if you persist in this approach you really need to find and read them if you haven't already.

Answer 2

1. Use a semaphore in your queue to indicate whether there are elements ready to be processed.
2. Every time you add an item, call ::ReleaseSemaphore to increment the count associated with the semaphore
3. In the loop in your thread process, call ::WaitForSingleObject() on the handle of your semaphore object -- you can give that wait a timeout so that you have an opportunity to know that your thread should exit. Otherwise, your thread will be woken up whenever there's one or more items for it to process, and also has the nice side effect of decrementing the semaphore count for you.

Answer 3

If you haven't read it, you should devour Herb Sutter's Effective Concurrency series which covers this topic and many many more.

Answer 4

Use condition variables to implement a producer/consumer queue - example code here.

If you need to support earlier versions of Windows you can use the condition variable in Boost. Or you could build your own by copying the Windows-specific code out of the Boost headers, they use the same Win32 APIs under the covers as you would if you build your own.

Answer 5

Why not just use the existing thread pool? Let Windows manage all of this.

Answer 6

1. You can use windows threadpool!
2. Or you can use api call WaitForSingleObject or WaitForMultipleObjects.
3. Use at least SwitchToThread api call when thread is workless.

Answer 7

If TaskList has some kind of wait_until_not_empty method then use it. If it does not then one Sleep(1000) (or some other value) may just do the trick. Proper solution would be to create a wrapper around TaskList that uses an auto-reset event handle to indicate if list is not empty. You would need to reinvent current methods for pop/push, with new task list being the member of new class:

WaitableTaskList::WaitableTaskList()
{
  // task list is empty upon creation
  non_empty_event = CreateEvent(NULL, FALSE, FALSE, NULL);
}

Task* WaitableTaskList::wait_and_pop_front(DWORD timeout)
{
  WaitForSingleObject(non_empty_event, timeout);
  // .. handle error, return NULL on timeout

  Task* result = task_list.pop_front();

  if (!task_list.empty())
    SetEvent(non_empty_event);

  return result;
}

void WaitableTaskList::push_back(Task* item)
{
  task_list.push_back(item);
  SetEvent(non_empty_event);
}

You must pop items in task list only through methods such as this wait_and_pop_front().

EDIT: actually this is not a good solution. There is a way to have non_empty_event raised even if the list is empty. The situation requires 2 threads trying to pop and list having 2 items. If list becomes empty between if and SetEvent we will have the wrong state. Obviously we need to implement syncronization as well. At this point I would reconsider simple Sleep again :-)