Why does this window subclassing code crash?

Question

I am trying to subclass the window that currently has focus. I do this by monitoring for HCBT_ACTIVATE events using a CBT hook, and set and unset the WndP

Answer 1

Your problems hinge on several fronts:

• UnHookWindowsHook does not unload injected dlls, all it does is remove the hook proc. If the dlls need to be unloaded its up to them to invent some kind of unloading mechanism.
• SetWindowLongPtr typically fails when called from a process other than the process that owns the window.

The nett result of this is, its very difficult to safely remove windows hooks. First thing, your OldWindowProc pointer should not be stored in the shared data area. Next, in order to remove the subclass, you need to be able to co-erce the (currently) subclassed process to perform the un-subclassing.

What you could do is, first, register a new unique message id and place it in your shared area using RegisterWindowMessage. WM_REMOVE_HOOK.

UINT idWM_REMOVE_HOOK = RegisterWindowMessage("WM_REMOVE_HOOK");

Now, whenever you need to remove a hook,

SendMessage(hWndSubClass,idWM_REMOVE_HOOK,0,0);

In your subclass proc:

if(uMsg == WM_DESTROY || uMsg == idWM_REMOVE_HOOK)
{
  Unsubclass(hwnd);
}

Remove the call to UnSubClass in DLL_PROCESS_DETATCH. Its a dangerous race condition thats going to cause your dll being unloaded in some random process to trash the hook data of a potentially valid hook in another process.