What are practical limits on the number of FileSystemWatcher instances a server can handle?

Question

I have a windows service that is currently instantiating about a dozen FileSystemWatcher instances to monitor shared folders across the corporate network for fi

Answer 1

FileSystemWatcher under the cover uses ReadDirectoryChangesW http://msdn.microsoft.com/en-us/library/windows/desktop/aa365465(v=vs.85).aspx. This is a reasonably inexpensive operation which is just a read from the directory that completes on a change.

The results are stored in a kernel buffer before they are copied into your own memory buffer of FileSystemWatcher.

That's the two OS resources to take into consideration, the Handle created by the call to CreateFile by FileSystemWatcher, and the 8KB (default) buffer size in the Kernel for each FileSystemWatcher object which takes away from your system's Kernel Paged and None-Paged Pools.

Your FileSystemWatchers are essentially competing for these three resources.

1. CPU time to process the changes
2. Handles on the system
3. Page Pool

You're unlikely to hit a problem with (2). Likely to hit a problem with (3) on a power system (loads of CPU) running x86. Otherwise (1) will be your limit.

Handles

Handles are exhaustible (specially on x86), more on this here, http://blogs.technet.com/b/markrussinovich/archive/2009/09/29/3283844.aspx

But at 16million+ handles (even on x86) before you run out, for your intententions, I'd think of it as an infinite resource. You'll exhaust the CPU processing changes well before you hit any OS limit.

Page/Non-Paged Pools

Page/Non-Paged Pools can be seen in task manager. On x86 they are very finite. More here, http://msdn.microsoft.com/en-us/library/windows/desktop/aa366778(v=vs.85).aspx#memory_limits

CPU

You'll see loads of anecdotal evidence that when this is exhausted, FileSystemWatcher sort of stops working. Some directory changes get reported, some don't, and inevitable on large implementations of FileSystemWatcher you end up having to detect these occassions and do a directory listing yourself, or do it on a polling bases.

Notes

If you're implementing a load of FileSystemWatchers watch out for;

1. Buffer over runs
2. Buffer size greater than 64KB on network paths.

More on good coding practice for this object here, http://bytes.com/topic/visual-basic-net/answers/536125-filesystemwatcher-across-network#post2092018