Buffer overrun during Garbage Collection: psychic debugging request

这一生的挚爱 提交于 2019-12-04 13:53:12

Unlike the v2 CLR, the v4 CLR was built with the Microsoft secure CRT extension enabled. Which include checks that, at function exit, the "stack canary" didn't get overwritten. Enabled by the /GS compiler option.

The likely end of your program in the previous version would have been a Fatal Execution Engine Exception, triggered by the access violation that would have been raised when the function tries to return and the return address got corrupted. It now catches the problem sooner. And more reliably, that corrupted return address could by accident point to valid code. What happens next if that's the case is usually truly undiagnosable. And exploitable.

But the root cause is the same, the GC heap getting corrupted.

Looks like a memory corruption to me; the library is likely using unsafe and/or unmanaged memory or pinned memory... or maybe it is not pinning the correct bits of memory, or unpinning them too early?

As for:

Running in VS (debug or release) stops it happening at all (or at least delays it longer than I'm prepared to wait)

This is because processes created by a debugger use a different heap (even if you are running in release mode); using this alternate heap is a known source of heisenbugs when dealing with random memory corruption (I have not found many sources on this point however; I thought it was on Raymond Chen blog somewhere but I only found this)

EDIT: reference found! From MSDN:

Processes that the debugger creates (also known as spawned processes) behave slightly differently than processes that the debugger does not create.
Instead of using the standard heap API, processes that the debugger creates use a special debug heap. You can force a spawned process to use the standard heap instead of the debug heap by using the _NO_DEBUG_HEAP environment variable or the -hd command-line option.

My best guess is then: the C++ library corrupts some memory. The GC comes, finds the heap corrupted, crash. OR: the C++ library does forget to pin the memory it is using as a buffer for images. The GC comes, move the memory. The C++ library does not know, writes to a now invalid pointer, causing corruption. The GC comes again, start to work on the now corrupted memory, crash

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!