I previously posted this guide in response to another question, but that question and my answer appear to have been deleted. It's not for the faint of heart:
- Install the Debugging Tools for Windows (Available as part of the Windows SDK) on the server
When the application has been running for a while, use adplus to capture a memory dump of the process (It's useful to use something such as Process Explorer to find the correct process ID to dump):
ADPLUS -hang -p <process id> -o .
This will create a directory containing the memory dump. You can now use windbg, and open the dump file (File -> Open Crash Dump...)
The joys of unmanaged code now appear. But you use something called Son of Strike, which understands .NET code, to see what objects are allocated. First you load SOS:
.loadby sos mscorwks
And then you ask it to examine the managed heap:
!dumpheap -stat
This generally spews a ton of output, but there are two columns showing the number of instances and the amount of memory being consumed, by type. Some types you expect to see a lot of (e.g. String), but if, say, there are thousands of instances of one of your own types, you might be leaking these objects somehow. One that's caught me in the past is hooking up an event handler in an object to a static event in the application - that event then has a live reference to every one of those objects.
I can never remember how most of this works, and generally refer to this cheat sheet for SOS
Tess Ferrandez has a good blog which sometimes covers .NET debugging using the unmanaged debuggers
E.g. a post from last May, detailing a potential problem if you use XmlSerializers with a non-default constructor.
