Loading an EXE as a DLL, local vftable

你说的曾经没有我的故事 提交于 2019-11-29 20:22:43

问题


I have an exe named test.exe which is usually used as a stand-alone application. I want to use this exe as a module (a dll) inside another application, app.exe.

The code in test.exe does something really simple like:

void doTest()
{
    MyClass *inst = new MyClass();
    inst->someMethod();
}

Where someMethod() is virtual and MyClass has a virtual d'tor.
doTest() is exported from test.exe and thus a lib called test.lib is created
app.exe is linked with this lib to statically load test.exe when it starts.

When I'm running test.exe stand-alone it runs just fine but when I'm running it loaded from within app.exe it crashes.
Stepping into the code with the debugger revealed that the crash is in the call to the virtual method. It turns out that the vftable somehow goes bad.

After some investigations it turns out that when the code inside the constructor of MyClass is running , the vftable is one thing but when the call to new returns it is replace with something else called a "local vftable". I found this obscure discussion about why this is.

After about a day of debugging it occurred to me that the pointers in this "local vftable" are the same in both cases, when test.exe is stand alone and whenit is loaded as a module. This can't be right because test.exe is loaded into a different address...
To test this theory I changed the loading address in the linker options to the one where test.exe is loaded when it is in app.exe and now, lo and behold, everything works.

Obviously, this is not a permanent solution because next time this randomly selected address may be occupied and the same problem will occur again.

So my question: Why is this "local vftable" tied to the static loading address of the exe? is loading an exe as a module a bad thing? why does the exe assume it is loaded to its static address?

Just for context: this is all done with MSVC 2008, Windows XP x64.


回答1:


VC++ strips out reloc info from .exes by default because normally they don't need to be relocatable.

You can force it to retain the reloc info with /fixed:no. See: http://msdn.microsoft.com/en-us/library/w368ysh2.aspx




回答2:


The workaround I ended up using is to simply add a compile configuration and compile the exe as a real dll instead of forcing it to act like one.

using /fixed:no didn't solve the problem for some reason.

Another difference I between exes and DLLs is that the entry point is different. a DLL's entry point is DllMain where as an exe has its entry point in the CRT which eventually calls main() or WinMain().



来源:https://stackoverflow.com/questions/1865631/loading-an-exe-as-a-dll-local-vftable

标签
易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!