Is it really that expensive to increase FileSystemWatcher.InternalBufferSize?

Question

I\'m using a FileSystemWatcher to monitor changes in a folder, but as soon as I have more than a few hundred modifications in a short time, I miss some of them

Answer 1

The memory in which the buffer gets allocated is certainly a precious resource. Windows will not deal with exhausting the memory pool well, drivers will start to fail at random. The size of the pool is dynamically set (but can be changed) and depends on the amount of available RAM.

The default buffer size that FSW asks for is 8192 bytes. Not much on modern machines. The underlying winapi function will not allow you to ask for more than 64KB. An entry is the buffer is 12 bytes plus the length of the file path times two. So worse case is 8192 / (12 + 260*2) = 15 notifications before the buffer runs out. That should work in most circumstances without much trouble, unless you monitor an entire drive or have very high disk traffic in the directory you are watching. In which case asking for a bigger buffer is fair. There is no golden formula, be sure to implement the FileSystemWatcher.Error event so you know that you've got a buffer problem.

In most practical cases you need to deal with the FSW events carefully. They will be raised while a process still has a lock on a file. So doing things like opening or copying the file are troublesome. You deal with that by putting the notifications on a thread-safe queue and use another thread to try to acquire a lock on the file, repeatedly if necessary. Such a queue is now automatically also a very good way to quickly empty the buffer. The only thing you've got to watch for now is that the queue doesn't blow up beyond reasonable proportions that will make your program crash with OOM.