Increasing MAXIMUM_WAIT_OBJECTS for WaitforMultipleObjects

Question

What is the simplest way to wait for more objects than MAXIMUM_WAIT_OBJECTS? MSDN lists this:

• Create a thread to wait on MAXIMUM_WAIT_OBJEC

Answer 1

I encountered this limitation in WaitForMultipleObjects myself and came to the conclusion I had three alternatives:

• OPTION 1. Change the code to create separate threads to invoke WaitForMultipleObjects in batches less than MAXIMUM_WAIT_OBJECTS. I decided against this option, because if there are already 64+ threads fighting for the same resource, I wanted to avoid creating yet more threads if possible.
• OPTION 2. Re-implement the code using a different technique (IOCP, for example). I decided against this too because the codebase I am working on is tried, tested and stable. Also, I have better things to do!
• OPTION 3. Implement a function that splits the objects into batches less than MAXIMUM_WAIT_OBJECTS, and call WaitForMultipleObjects repeatedly in the same thread.

So, having chosen option 3 - here is the code I ended up implementing ...

class CtntThread
{
    public: 
        static DWORD WaitForMultipleObjects( DWORD, const HANDLE*, DWORD millisecs );
};

DWORD CtntThread::WaitForMultipleObjects( DWORD count, const HANDLE *pHandles, DWORD millisecs )
{
    DWORD retval = WAIT_TIMEOUT;

    // Check if objects need to be split up. In theory, the maximum is
    // MAXIMUM_WAIT_OBJECTS, but I found this code performs slightly faster
    // if the object are broken down in batches smaller than this.
    if ( count > 25 )
    {
        // loop continuously if infinite timeout specified
        do
        {
            // divide the batch of handles in two halves ...
            DWORD split = count / 2;
            DWORD wait = ( millisecs == INFINITE ? 2000 : millisecs ) / 2;
            int random = rand( );

            // ... and recurse down both branches in pseudo random order
            for ( short branch = 0; branch < 2 && retval == WAIT_TIMEOUT; branch++ )
            {
                if ( random%2 == branch ) 
                {
                    // recurse the lower half
                    retval = CtntThread::WaitForMultipleObjects( split, pHandles, wait );
                }
                else
                {
                    // recurse the upper half
                    retval = CtntThread::WaitForMultipleObjects( count-split, pHandles+split, wait );
                    if ( retval >= WAIT_OBJECT_0 && retval < WAIT_OBJECT_0+split ) retval += split;
                }
            }
        }
        while ( millisecs == INFINITE && retval == WAIT_TIMEOUT );
    }
    else
    {
        // call the native win32 interface
        retval = ::WaitForMultipleObjects( count, pHandles, FALSE, millisecs );
    }

    // done
    return ( retval );
}

Answer 2

Have a look here.

If you need to wait on more than MAXIMUM_WAIT_OBJECTS handles, you can either create a separate thread to wait on MAXIMUM_WAIT_OBJECTS and then do a wait on these threads to finish. Using this method you can create MAXIMUM_WAIT_OBJECTS threads each of those can wait for MAXIMUM_WAIT_OBJECTS object handles.

Answer 3

If you find yourself waiting on tons of objects you might want to look into IO Completion Ports instead. For large numbers of parallel operations IOCP is much more efficient.

And the name IOCP is misleading, you can easily use IOCP for your own synchronization structures as well.