I have an app with a number of worker threads, one for each core. On a modern 8 core machine, I have 8 of these threads. My app loads a lot of plugins, which also have their
Reducing $MAXSTACKSIZE won't work because Windows will always align thread stack to 1Mb (?).
One (possible?) way to prevent fragmentation is to reserve (not alloc!) virtual memory (with VirtualAlloc) before creating threads. And release it after the threads are running. This way Windows cannot use the reserved space for the threads so you will have some continuous memory.
Or you could make your own memory manager for large photo's: reserve a lot virtual memory and alloc memory from this pool by hand. (you need to maintain a list of used and used memory yourself).
At least, that's a theory, don't know if it really works...