Since you asked for pointers to books... As far as Windows debugging goes, John Robbins has several editions of a good book on Windows debugging:
Debugging Applications for Microsoft .NET and Microsoft Windows
Note that the most recent edition (Debugging Microsoft .NET 2.0 Applications) is .NET only, so you might want an older one (like in the first link) if you want native code debugging (it covers both .NET and native).
